System and method for assessing biological tissue

ABSTRACT

A system for assessing biological tissue is disclosed. The system contains an illumination hardware arrangement comprising transmission and sensing hardware, the illumination hardware arrangement configured to inspect a biological tissue using at least two modes from a group containing: a three dimensional stereo imaging mode; a fluorescence imaging mode; a reflectance imaging mode; and a thermal imaging mode; and processing hardware configured to operate the illumination hardware arrangement according to a protocol comprising inspection settings of the at least two modes, wherein the processing hardware receives scan results for the at least two modes from the illumination hardware arrangement and identifies attributes of the biological tissue by constructing a three dimensional dataset from the scan results for the at least two modes and analyzing the three dimensional dataset.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation-in-part of U.S. patent applicationSer. No. 17/681,568 titled “Apparatus and Method for MultimodeAnalytical Sensing of Items Such as Food” filed Feb. 25, 2022, which isincorporated herein by reference in its entirety. The U.S. patentapplication Ser. No. 17/681,568 is a continuation of U.S. patentapplication Ser. No. 16/358,531 titled “Apparatus and Method forMultimode Analytical Sensing of Items Such as Food” filed Mar. 19, 2019,now issued U.S. Pat. No. 11,280,777, which is incorporated herein byreference in its entirety. The U.S. patent application Ser. No.16/358,531 is based upon provisional patent application of U.S. Ser. No.62/645,514 filed Mar. 20, 2018, which is incorporated herein byreference in its entirety. This nonprovisional patent application isbased upon provisional patent application, U.S. Ser. No. 63/256,571filed Oct. 16, 2021, which is hereby incorporated by reference in itsentirety.

FIELD

The present subject matter, in general is directed to a system andmethod for assessing biological tissue and more particularly, isdirected to a non-invasive imaging system and method for assessingbiological tissue.

BACKGROUND

Diabetic wounds in the US affect more than 8.2 million patients annuallywith an estimated economic burden of $28 billion (USD). Diabetic woundsare a growing concern due to the obesity epidemic and the attendant risein type 2 diabetes. Ulcers are a common complication of diabetes. Up to34% of people diagnosed with diabetes will develop a lower extremityulcer at some point in their lifetime. Standard treatment for diabeticwounds includes debridement of necrotic tissue, infection control withtopical dressings, mechanical off-loading, and management of bloodglucose levels. If wounds do not adequately heal with standard treatmentfor diabetic care, additional interventions may be required includingvenous surgical intervention. Treatment selection depends largely uponthe treating provider's subjective impression of the state of the wound,the non-invasive assessment of vascularity by techniques including butnot limited to ankle/brachial indices, oximetry, toe pressures,non-invasive Doppler vascular studies, and whether a wound seems to bemaking progress or not based on change in size, which can be difficultto measure accurately in irregularly shaped wounds and which can varydepending on the person measuring them.

Unfortunately, features with significance to wound development may beunintentionally omitted due to the subjectivity of a clinician'sdiagnosis, and abnormalities may be revealed only after irreversiblechanges have occurred during later stages of diabetes. Current treatmentof non-healing wounds involves pressure reduction to prevent furtherischemia, improved nutrition, and bedside wound care with topicalantibiotics, debridement, and dressings to limit infection. A majorobstacle in improving wound-healing treatment regimens is a lack ofobjective physiological and biochemical markers able to be used toassess status of the wound and the effectiveness of treatment.

Thus, there is a need to better understand wound physiology usingspecialized tools that have been designed to measure underlying tissuechanges and evaluate wound status to guide wound dressing selection andfrequency of change.

SUMMARY

According to some embodiments, there is provided a biological sampleinspection apparatus, comprising an illumination hardware arrangementcomprising transmission and sensing hardware, the illumination hardwarearrangement configured to inspect a biological sample using at least twomodes from a group comprising a fluorescence imaging mode, a reflectanceimaging mode, a scattering imaging mode, and a Raman imaging mode, andprocessing hardware configured to operate the illumination hardwarearrangement according to a protocol comprising inspection settings ofthe at least two modes. According to some embodiments, the processinghardware receives scan results from the illumination hardwarearrangement and identifies attributes of the biological sample. Theprocessing hardware may be configured to employ the attributes of atleast one biological sample to alter the protocol.

According to some embodiments, there is provided a method for inspectingat least one biological sample, comprising determining a plurality ofinspection modes for inspecting the at least one biological sample usinga multimode inspection apparatus, determining an inspection protocol forinspecting the at least one biological sample, wherein the inspectionprotocol comprises inspection settings for the plurality of inspectionmodes, inspecting at least one biological sample using the multimodeinspection apparatus according to the protocol, and altering theprotocol based on inspection results for multiple biological samples.

According to some embodiments, there is provided a biological sampleinspection apparatus configured to inspect a biological sample forissues, comprising illumination hardware comprising transmission andsensing hardware configured to illuminate and sense attributes of thebiological sample, the illumination hardware configured to inspect thebiological sample using multiple inspection configurations from at leastone of a fluorescence imaging mode, a reflectance imaging mode, ascattering imaging mode, and a Raman imaging mode, and processinghardware configured to operate the illumination hardware according to aprotocol comprising inspection settings for the multiple inspectionconfigurations, wherein the processing hardware receives scan resultsfrom the illumination hardware and identifies attributes of thebiological sample. The processing hardware may be configured to employthe attributes of at least one biological sample and alter the protocolbased on the attributes of the one biological sample.

Generally speaking, pursuant to the various embodiments, according toone aspect, a system for assessing biological tissue is presentlydisclosed. The system comprises an illumination hardware arrangementcomprising transmission and sensing hardware, the illumination hardwarearrangement configured to inspect a biological tissue using at least twomodes from a group of modes. The group of modes are a three dimensionalstereo imaging mode; a fluorescence imaging mode; a reflectance imagingmode; and a thermal imaging mode. The system further comprisesprocessing hardware configured to operate the illumination hardwarearrangement according to a protocol comprising inspection settings ofthe at least two modes, wherein the processing hardware receives scanresults for the at least two modes from the illumination hardwarearrangement and identifies attributes of the biological tissue byconstructing a three dimensional dataset from the scan results for theat least two modes and analyzing the three dimensional dataset.According to another aspect, the biological tissue being assessed by thesystem is a wound on a patient's skin. According to another aspect, theprocessing hardware of the system comprises a processor, at least onetrained artificial intelligence module, and at least one classifier.According to another aspect, the biological tissue being assessed by thesystem is a burn on a patient's skin. According to another aspect, theprotocol of the system is determined in part based on an identificationof particular attributes expected to be associated with the biologicaltissue when examined using the at least two modes. According to anotheraspect, the system determines presence of infection associated with thebiological tissue. According to another aspect, the attributesidentified by the system are infection associated with the biologicaltissue. According to another aspect, the attributes identified by thesystem are metabolic biomarkers associated with the biological tissue.According to another aspect, the illumination hardware of the systemcomprises one or more pulsating light sources to reduce ambient lightsources for the at least two modes. According to another aspect, theattributes identified by the system are collagen associated with thebiological tissue. According to another aspect, the attributesidentified by the system are oxygenation associated with the biologicaltissue. According to another aspect, the system identifies treatmentbased on the three dimensional dataset. According to another aspect, thesystem identifies billing code based on the three dimensional dataset.According to another aspect, the system identifies treatment based onthe three dimensional dataset. According to another aspect, the systemclassifies the wound based on the three dimensional dataset. Accordingto another aspect, the system classifies the burn based on the threedimensional dataset.

These and other advantages of the present invention will become apparentto those skilled in the art from the following detailed description ofthe invention and the accompanying drawings.

BRIEF DESCRIPTION OF THE FIGS

FIG. 1 depicts a front perspective view of a device according to someembodiments presently disclosed;

FIG. 2 depicts a rear perspective view of the device in FIG. 1 ;

FIG. 3 depicts a block diagram of a device according to some embodimentspresently disclosed;

FIG. 4 depicts biological sample and reflectance spectral signatures ofthe biological sample;

FIG. 5 depicts schematic of electron transport chain;

FIG. 6 a depicts tissue NADH and FAD fluorescent images;

FIG. 6 b depicts tissue redox ratio histograms;

FIG. 7 a depicts tissue vasculature extracted from fluorescence imaging;

FIG. 7 b depicts color coded vascular diameter of the tissuevasculature;

FIG. 8 a depicts 3D image scanner of foot;

FIG. 8 b depicts thermal image of a foot overlay on 3D foot model;

FIG. 9 depicts a conceptual representation of fusion artificialintelligence according to some embodiments presently disclosed;

FIG. 10 depicts biological sample and fluorescence spectral signaturesof the biological sample excited by two illumination wavelengths;

FIG. 11 depicts artificial intelligence model according to someembodiments presently disclosed;

FIG. 12 depicts an alternative embodiments of the fusion artificialintelligence processing according to some embodiments presentlydisclosed;

FIG. 13 depicts conceptual overview of operation according to someembodiments presently disclosed;

FIG. 14 depicts a method according to some embodiments presentlydisclosed;

FIG. 15 depicts another method according to some embodiments presentlydisclosed;

FIG. 16 depicts another method according to some embodiments presentlydisclosed;

FIG. 17 depicts another method according to some embodiments presentlydisclosed;

FIG. 18 depicts different classification grades for a wound; and

FIG. 19 depicts another method according to some embodiments presentlydisclosed.

In the following description, like reference numbers are used toidentify like elements. Furthermore, the drawings are intended toillustrate major features of exemplary embodiments in a diagrammaticmanner. The drawings are not intended to depict every feature of everyimplementation nor relative dimensions of the depicted elements, and arenot drawn to scale.

DETAILED DESCRIPTION

In the following description, numerous specific details are set forth toclearly describe various specific embodiments disclosed herein. Oneskilled in the art, however, will understand that the presently claimedinvention may be practiced without all of the specific details discussedbelow. In other instances, well known features have not been describedso as not to obscure the invention.

Also, it is to be understood that the phraseology and terminology usedherein is for the purpose of description and should not be regarded aslimiting. The use of “including,” “comprising,” or “having” andvariations thereof herein is meant to encompass the items listedthereafter and equivalents thereof as well as additional items. Unlesslimited otherwise, the terms “connected,” “coupled,” and “mounted,” andvariations thereof herein are used broadly and encompass direct andindirect connections, couplings, and mountings. In addition, the terms“connected” and “coupled” and variations thereof are not restricted tophysical or mechanical connections or couplings.

In addition, it should be understood that embodiments of the inventioninclude both hardware and electronic components or modules that, forpurposes of discussion, may be illustrated and described as if themajority of the components were implemented solely in hardware. However,one of ordinary skill in the art, and based on a reading of thisdetailed description, would recognize that, in at least one embodiment,the electronic based aspects of the invention may be implemented insoftware. As such, it should be noted that a plurality of hardware andsoftware-based devices, as well as a plurality of different structuralcomponents may be utilized to implement the invention. Furthermore, andas described in subsequent paragraphs, the specific mechanicalconfigurations illustrated in the drawings are intended to exemplifyembodiments of the invention and that other alternative mechanicalconfigurations are possible.

According to some embodiments, presently disclosed system and method maybe used to evaluate biological tissue, such as, for example, burn(s) onpatient's skin, subdermal tissue, bone tissue, wound on patient's skin,wound on patient's skin caused by diabetes, and/or any other aspects ofpatient's organs. According to some embodiments, presently disclosedsystem and method may be used to detect and/or identify one or moreaspects of the biological tissue. For example, the presently disclosedsystem and method may be used to: analyze collagen content in apatient's skin, analyze collagen content in the patient's subdermaltissue, identify presence of an infection on the patient's skin,identify type of infection present on the patient's skin, identifypresence of an infection in the patient's subdermal tissue, identifytype of infection present in the patient's subdermal tissue, identifypresence of blood vessels for perfusion, identify oxygen content in thepatient's skin, identify oxygen content in the patient's subdermaltissue, identify size of a burn on patient's skin, identify size of awound on patient's skin, identify presence of necrotic tissue onpatient's skin, identify presence of necrotic tissue in the patient'ssubdermal tissue, identify presence of bone tissue in patient's wound,identify type of subdermal tissue that is visible in patient's wound,identify presence of fat tissue in patient's wound, analyze collagencontent in the biological tissue, analyze collagen content in thebiological tissue, identify presence of an infection on the biologicaltissue, identify type of infection present on the biological tissue,identify presence of an infection in the biological tissue, identifytype of infection present in the biological tissue, identify presence ofblood vessels for perfusion in the biological tissue, identify oxygencontent in the biological tissue, identify size of the biologicaltissue, identify presence of necrotic tissue on the biological tissue,identify presence of necrotic tissue in the biological tissue, identifypresence of bone tissue in the biological tissue, identify type ofsubdermal tissue that is visible in the biological tissue, and/oridentify presence of fat tissue in the biological tissue. According tosome embodiments, presently disclosed system and method may classifyburns as electrical burns or chemical burns based on differentbiomarkers on the biological tissue.

Referring to FIGS. 1-2 , a device 10 is shown according to someembodiments presently disclosed. Referring to FIG. 3 , a block diagram20 is shown according to some embodiments presently disclosed. The blockdiagram 20 depicts some of the components of the device 10 and how theycommunicate with one another. According to some embodiments presentlydisclosed, the device 10 is a handheld device. According to someembodiments presently disclosed, the device 10 is part of non-invasiveimaging system and method for assessing biological tissue.

According to some embodiments presently disclosed, an operator (i.e.user, medical professional, technician) uses the device 20 to collectimages of the biological tissue.

According to some embodiments presently disclosed, the device 10comprises a housing 22 with one or more handles 23. According to someembodiments, the housing 22 of the device 10 comprises additionalmaterials for ruggedization or to provide drop/impact resistance.

According to some embodiments presently disclosed, the device 10comprises a memory 74 (which may comprise one or more computer readablestorage mediums). The memory 74 may comprise high-speed random-accessmemory and/or non-volatile memory, such as one or more magnetic diskstorage devices, flash memory devices, or other non-volatile solid-statememory devices. Access to memory 74 by other components of the device10, such as one or more system processor modules 65 and a peripheralsinterface, may be controlled by a memory controller (not shown).

According to some embodiments presently disclosed, the device 10comprises one or more system processor modules 65. The one or moresystem processor modules 65 run or execute various software programsand/or sets of instructions stored in memory 74 to perform variousfunctions for the device 10 and to process data. The system processormodule 65 may also comprise orientation sensors, motion sensors, globalpositioning systems, wireless communication systems such as WiFi orBluetooth systems, cellular network communications systems such 4G, LTEor 5G or similar systems. The system processor module 65 may use thesesystems to communicate with a device server 90 or it may communicatewith the device server via a wired connection through a peripheralinterface. The system processor module 65 may also use these systems tocommunicate with other wireless devices such as cell phones, tablets,smart glasses, other inspection devices or other smart displays as wellas RFID systems, barcode readers, fingerprint readers, etc. According tosome embodiments, some or all of these components may be implemented ona single chip. According to some embodiments, some or all of thesecomponents may be implemented on separate chips.

According to some embodiments presently disclosed, the device 10comprises an audio circuitry 110, a speaker 111, and a microphone 113.The audio circuitry 110, the speaker 111, and the microphone 113 providean audio interface between a user (i.e. operator) and the device 10. Theaudio circuitry 110 receives audio data, converts the audio data to anelectrical signal, and transmits the electrical signal to the speaker111. The speaker 111 converts the electrical signal to human-audiblesound waves. The audio circuitry 110 also receives electrical signalsconverted by the microphone 113 from sound waves. The audio circuitry110 converts the electrical signal to audio data and transmits the audiodata to one or more system processor modules 65 for processing. Audiodata may be retrieved from and/or transmitted to memory 74. The audiocircuitry 110 may also comprise a headset/speaker jack (not shown). Theheadset jack provides an interface between the audio circuitry 110 andremovable audio input/output peripherals, such as speaker, output-onlyheadphones and/or a headset with both output (e.g., a headphone for oneor both ears) and input (e.g., a microphone).

According to some embodiments presently disclosed, the device 10comprises a display 70. The display 70 may be a touch-sensitive display70. The touch-sensitive display 70 is sometimes called a “touch screen”for convenience, and may also be known as or called a touch-sensitivedisplay system. In one embodiment, the touch-sensitive touch screen 70provides an input interface and an output interface between the device10 and the user. The touch screen 70 is configured to implement virtualor soft buttons and one or more soft keyboards. A display controllerreceives and/or sends electrical signals from/to the touch screen 70.The touch screen 70 displays visual output to the user. The visualoutput may include graphics, text, icons, video, and any combinationthereof (collectively termed “graphics”). In some embodiments, some orall of the visual output may correspond to user-interface objects,further details of which are described below.

The touch screen 70 has a touch-sensitive surface, sensor or set ofsensors that accepts input from the user based on haptic and/or tactilecontact. The touch screen 70 and the display controller (along with anyassociated modules and/or sets of instructions in memory 74) detectcontact (and any movement or breaking of the contact) on the touchscreen 70 and converts the detected contact into interaction withuser-interface objects (e.g., one or more soft keys, icons, web pages orimages) that are displayed on the touch screen. In one embodiment, apoint of contact between a touch screen 70 and the user corresponds to afinger of the user.

The touch screen 70 may use LCD (liquid crystal display) technology, orLPD (light emitting polymer display) technology, although other displaytechnologies may be used in other embodiments. The touch screen 70 andthe display controller may detect contact and any movement or breakingthereof using any of a plurality of touch sensing technologies now knownor later developed, including but not limited to capacitive, resistive,infrared, and surface acoustic wave technologies, as well as otherproximity sensor arrays or other elements for determining one or morepoints of contact with the touch screen 70.

A touch-sensitive display in some embodiments of the touch screen 70 maybe analogous to the multi-touch sensitive tablets described in thefollowing U.S. Pat. No. 6,323,846 (Westerman et al.), U.S. Pat. No.6,570,557 (Westerman et al.), and/or U.S. Pat. No. 6,677,932(Westerman), and/or U.S. Patent Publication 2002/0015024A1, each ofwhich is hereby incorporated by reference in its entirety. However, atouch screen 70 displays visual output from the portable device 10,whereas touch sensitive tablets do not provide visual output.

A touch-sensitive display in some embodiments of the touch screen 70 maybe as described in the following applications: (1) U.S. patentapplication Ser. No. 11/381,313, “Multipoint Touch Surface Controller,”filed May 2, 2006; (2) U.S. patent application Ser. No. 10/840,862,“Multipoint Touchscreen,” filed May 6, 2004; (3) U.S. patent applicationSer. No. 10/903,964, “Gestures For Touch Sensitive Input Devices,” filedJul. 30, 2004; (4) U.S. patent application Ser. No. 11/048,264,“Gestures For Touch Sensitive Input Devices,” filed Jan. 31, 2005; (5)U.S. patent application Ser. No. 11/038,590, “Mode-Based Graphical UserInterfaces For Touch Sensitive Input Devices,” filed Jan. 18, 2005; (6)U.S. patent application Ser. No. 11/228,758, “Virtual Input DevicePlacement On A Touch Screen User Interface,” filed Sep. 16, 2005; (7)U.S. patent application Ser. No. 11/228,700, “Operation Of A ComputerWith A Touch Screen Interface,” filed Sep. 16, 2005; (8) U.S. patentapplication Ser. No. 11/228,737, “Activating Virtual Keys Of ATouch-Screen Virtual Keyboard,” filed Sep. 16, 2005; and (9) U.S. patentapplication Ser. No. 11/367,749, “Multi-Functional Hand-Held Device,”filed Mar. 3, 2006. All of these applications are incorporated byreference herein in their entirety.

The touch screen 70 may have a resolution of 100 dpi. to 350 dpi. Theuser may make contact with the touch screen 70 using any suitable objector appendage, such as a stylus, a finger, and so forth. In someembodiments, the user interface is designed to work primarily withfinger-based contacts and gestures, which are much less precise thanstylus-based input due to the larger area of contact of a finger on thetouch screen. In some embodiments, the device translates the roughfinger-based input into a precise pointer/cursor position or command forperforming the actions desired by the user.

In addition to the touch screen 70, the device 10 may comprise atouchpad (not shown) for activating or deactivating particularfunctions. The touchpad is a touch-sensitive area of the device that,unlike the touch screen, does not display visual output. The touchpadmay be a touch-sensitive surface that is separate from the touch screen70 or an extension of the touch-sensitive surface formed by the touchscreen.

The one or more system processor modules 65 may be configured tocommunicate with the smart display 70 to provide information to the userduring an inspection or to accept instructions from the operator duringan inspection. According to some embodiments, the smart display 70 maybe a passive device such as a touch screen display. According to someembodiments, the smart display 70 may be an active device with multipleprocessing and communication capabilities such as a smartphone ortablet. If the smart display 70 is an active device some of the systemsoftware functions may be shared between the one or more systemprocessor modules 65 and the smartphone or tablet. According to someembodiments, the smart display 70 is a smartphone.

The device 10 may also comprise a radio frequency (RF) circuitry 108.The RF circuitry 108 may be configured to receive and transmit RFsignals, also called electromagnetic signals. The RF circuitry 108converts electrical signals to/from electromagnetic signals andcommunicates with communications networks and other communicationsdevices via the electromagnetic signals. The RF circuitry 108 mayinclude circuitry for performing these functions, including but notlimited to an antenna system, an RF transceiver, one or more amplifiers,a tuner, one or more oscillators, a digital signal processor, a CODECchipset, a subscriber identity module (SIM) card, memory, and so forth.The RF circuitry 108 may communicate with networks, such as theInternet, also referred to as the World Wide Web (WWW), an intranetand/or a wireless network, such as a cellular telephone network, awireless local area network (LAN) and/or a metropolitan area network(MAN), and other devices by wireless communication. The wirelesscommunication may use any of a plurality of communications standards,protocols and technologies, including but not limited to Global Systemfor Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE),high-speed downlink packet access (HSDPA), wideband code divisionmultiple access (W-CDMA), code division multiple access (CDMA), timedivision multiple access (TDMA), Bluetooth, Wireless Fidelity (Wi-Fi)(e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g and/or IEEE 802.11n),voice over Internet Protocol (VoIP), Wi-MAX, a protocol for email (e.g.,Internet message access protocol (IMAP) and/or post office protocol(POP)), instant messaging (e.g., extensible messaging and presenceprotocol (XMPP), Session Initiation Protocol for Instant Messaging andPresence Leveraging Extensions (SIMPLE), and/or Instant Messaging andPresence Service (IMPS)), and/or Short Message Service (SMS)), or anyother suitable communication protocol, including communication protocolsnot yet developed as of the filing date of this document. According tosome embodiments, the radio frequency (RF) circuitry 108 allows thedevice 10 to communicate with a device server 90 and/or an externalserver 95.

The device 10 may also comprise a physical or virtual click wheel (notshow) and/or one or more controls 80 as an input control device. Theuser may navigate among and interact with one or more graphical objects(henceforth referred to as icons) displayed in the screen 70 by rotatingthe click wheel or by moving a point of contact with the click wheel(e.g., where the amount of movement of the point of contact is measuredby its angular displacement with respect to a center point of the clickwheel) or by activating the one or more controls 80. The click wheel mayalso be used to select one or more of the displayed icons. For example,the user may press down on at least a portion of the click wheel or anassociated button. User commands and navigation commands provided by theuser via the click wheel may be processed by an input controller as wellas one or more of the modules and/or sets of instructions in memory 74.For a virtual click wheel, the click wheel and click wheel controllermay be part of the touch screen 70 and the display controller,respectively. For a virtual click wheel, the click wheel may be eitheran opaque or semitransparent object that appears and disappears on thetouch screen display in response to user interaction with the device. Insome embodiments, a virtual click wheel is displayed on the touch screenof a portable multifunction device and operated by user contact with thetouch screen.

According to some embodiments presently disclosed, the device 10comprises a power system 75. The power system 75 powers variouscomponents of the device 10. The power system 75 may comprise a powermanagement system, one or more power sources (e.g., battery, alternatingcurrent (AC)), a recharging system, a power failure detection circuit, apower converter or inverter, a power status indicator (e.g., alight-emitting diode (LED)) and/or any other components associated withthe generation, management and distribution of power in portabledevices.

According to some embodiments presently disclosed, the device 10comprises an optical sensor 25. The optical sensor 25 of the device 10may be electrically coupled with an optical sensor controller. Theoptical sensor 25 may comprise charge-coupled device (CCD) orcomplementary metal-oxide semiconductor (CMOS) phototransistors. Theoptical sensor 25 receives light from the environment, projected throughone or more lens, and converts the light to data representing an image.In conjunction with an imaging module (also called a camera module), theoptical sensor 25 may capture visual media (i.e. still images or video).In some embodiments, the optical sensor 25 may be located on the frontof the device 10, opposite the touch screen display 70 on the back ofthe device 10, so that the touch screen display 70 may be used as aviewfinder for either still and/or video image acquisition. In someembodiments, the optical sensor 25 may be located on the back of thedevice 10 to capture image(s) of the user. In some embodiments, oneoptical sensor 25 may be located on the back of the device 10 andanother optical sensor 25 may be located on the front of the device 10.In some embodiments, the position of the optical sensor 25 may bechanged by the user (e.g., by rotating the lens and the sensor in thedevice housing) so that a single optical sensor 25 may be used alongwith the touch screen display to capture still and/or video image.

According to some embodiments presently disclosed, the device 10comprises an optical sensor 30. The optical sensor 30 of the device 10may be electrically coupled with an optical sensor controller. Theoptical sensor 30 may comprise charge-coupled device (CCD) orcomplementary metal-oxide semiconductor (CMOS) phototransistors. Theoptical sensor 30 receives light from the environment, projected throughone or more lens, and converts the light to data representing an image.In conjunction with an imaging module (also called a camera module), theoptical sensor 30 may capture visual media (i.e. still images or video).In some embodiments, the optical sensor 30 may be located on the frontof the device 10, opposite the touch screen display 70 on the back ofthe device 10, so that the touch screen display 70 may be used as aviewfinder for either still and/or video image acquisition. In someembodiments, the optical sensor 30 may be located on the back of thedevice 10 to capture image(s) of the user. In some embodiments, oneoptical sensor 30 may be located on the back of the device 10 andanother optical sensor 30 may be located on the front of the device 10.In some embodiments, the position of the optical sensor 30 may bechanged by the user (e.g., by rotating the lens and the sensor in thedevice housing) so that a single optical sensor 30 may be used alongwith the touch screen display to capture still and/or video image.

According to some embodiments presently disclosed, the optical sensor 25may comprise fluorescence imaging camera, 3D stereoscopic imagingcamera, thermal imaging camera, or speckle imaging camera. According tosome embodiments presently disclosed, the optical sensor 30 may comprisefluorescence imaging camera, 3D stereoscopic imaging camera, thermalimaging camera, or speckle imaging camera. According to some embodimentspresently disclosed, the device 10 may comprise fluorescence imagingcamera, 3D stereoscopic imaging camera 501, thermal imaging camera 502,and/or speckle imaging camera.

According to some embodiments presently disclosed, the optical sensor 25may comprise triple band pass filter (440 nm and 550 nm, and 700 nm).The triple band pass filters are configured to cut off the NADHexcitation wavelength to the optical sensor 25. According to someembodiments presently disclosed, the optical sensor 30 may comprisedouble band pass filter (520 nm and 700 nm). The double band passfilters are configured to cut off the NADH/FAD excitation wavelength tothe optical sensor 30.

According to some embodiments presently disclosed, the optical sensor 25is a color optical sensor. According to some embodiments presentlydisclosed, the optical sensor 25, when imaging under ambient light, mayact as a view finder for operators to position the system correctlyprior to biomarker measurements and for conventional wound dimensionmeasurements.

According to some embodiments presently disclosed, the device 10comprises a range finder to calibrate the field of view at each imagecapture distance for comparing wound dimensions across different imagesand over time.

According to some embodiments presently disclosed, the device 10 mayalso comprise one or more accelerometers 168 as shown in FIG. 3 . Theaccelerometer 168 may perform as described in U.S. Patent PublicationNo. 20050190059, “Acceleration-based Theft Detection System for PortableElectronic Devices,” and U.S. Patent Publication No. 20060017692,“Methods And Apparatuses For Operating A Portable Device Based On AnAccelerometer,” both of which are which are incorporated herein byreference in their entirety. Information may be displayed on the touchscreen display 70 in a portrait view or a landscape view based on ananalysis of data received from the one or more accelerometers 168.

According to some embodiments, the memory 74 may be configured to storeone or more software components as described below.

The memory 74 may be configured to store an operating system. Theoperating system (e.g., Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or anembedded operating system such as VxWorks) comprises various softwarecomponents and/or drivers for controlling and managing general systemtasks (e.g., memory management, storage device control, powermanagement, etc.) and facilitates communication between various hardwareand software components.

The memory 74 may be configured to store a system software. The systemsoftware may provide data storage for measurements and other informationthat are transferred from the device 10. The system software may providesystem management functions for managing the creation of jobs and tasklists that can be implemented using the device 10. The system softwaremay be configured to manage data storage and creation of jobs and tasklists for one or more devices 10 for an organization. The systemsoftware may comprise firmware software, analysis software, and userinterface software.

The memory 74 may also be configured to store a communication module.The communication module facilitates communication with other devicesover one or more external ports and also includes various softwarecomponents for handling data received by the RF circuitry 108 and/or theexternal port. In one embodiment, the external port (e.g., UniversalSerial Bus (USB), FIREWIRE, etc.) is configured for coupling directly toother devices or indirectly over a network (e.g., the Internet, wirelessLAN, etc.).

The memory 74 may be configured to store a contact/motion module. Thecontact/motion module is configured to detect contact with the touchscreen 70 (in conjunction with the display controller) and other touchsensitive devices (e.g., a touchpad or physical click wheel). Thecontact/motion module includes various software components forperforming various operations related to detection of contact, such asdetermining if contact has occurred, determining if there is movement ofthe contact and tracking the movement across the touch screen 74, anddetermining if the contact has been broken (i.e., if the contact hasceased). Determining movement of the point of contact may includedetermining speed (magnitude), velocity (magnitude and direction),and/or an acceleration (a change in magnitude and/or direction) of thepoint of contact. These operations may be applied to single contacts(e.g., one finger contacts) or to multiple simultaneous contacts (e.g.,“multitouch”/multiple finger contacts). The contact/motion module andthe display controller may also detect contact on a touchpad. Thecontact/motion module and the controller may further detect contact on aclick wheel.

The memory 74 may be configured to store a graphics module. The graphicsmodule comprises various known software components for rendering anddisplaying graphics on the touch screen 70, including components forchanging the intensity of graphics that are displayed. As used herein,the term “graphics” includes any object that can be displayed to a user,including without limitation text, web pages, icons (such asuser-interface objects including soft keys), digital images, videos,animations and the like.

The memory 74 may also be configured to store a text input module. Thetext input module, which may be a component of graphics module, providessoft keyboards for entering text in various applications that need textinput.

The memory 74 may be configured to store a GPS module. The GPS moduledetermines the location of the device and provides this information foruse in various applications (e.g., to camera module as picture/videometadata).

The memory 74 may be configured to store applications. The applicationsmay comprise one or more of the following modules (or sets ofinstructions), or a subset or superset thereof: a camera module forstill and/or video images; an image management module; a video playermodule; and/or online video module.

The applications may comprise additional modules (or sets ofinstructions). For example, other applications that may be stored inmemory 74 may include one or more of the following: a contacts module(sometimes called an address book or contact list); a telephone module;a video conferencing module; an e-mail client module; an instantmessaging (IM) module; a browser module; a calendar module; searchmodule; notes module; map module; word processing applications;JAVA-enabled applications; encryption; digital rights management; voicerecognition; and/or voice replication.

The camera module (in conjunction with, for example, touch screen 70,display controller, optical sensor(s) 25 and/or 30, optical sensorcontroller, contact module, graphics module, and image managementmodule) may be configured to capture still images or video (including avideo stream) and store them into memory 74, modify characteristics of astill image or video, or delete a still image or video from memory 74.

The image management module (in conjunction with, for example, touchscreen 70, display controller, contact module, graphics module, textinput module, and camera module) may be configured to arrange, modify orotherwise manipulate, label, delete, present (e.g., in a digital slideshow or album), and store still and/or video images.

The video player module (in conjunction with, for example, touch screen70, display controller, contact module, graphics module, audio circuitry110, and speaker 111) may be configured to display, present or otherwiseplay back videos (e.g., on the touch screen 70 or on an external,connected display via external port).

The online video module (in conjunction with, for example, touch screen70, display system controller, contact module, graphics module, audiocircuitry 110, speaker 111, RF circuitry 108,) may be configured toallow the user to access, browse, receive (e.g., by streaming and/ordownload), play back (e.g., on the touch screen 70 or on an external,connected display via external port), upload and/or otherwise manageonline videos in one or more file formats.

Each of the above identified modules and applications correspond to aset of instructions for performing one or more functions describedabove. These modules (i.e., sets of instructions) need not beimplemented as separate software programs, procedures or modules, andthus various subsets of these modules may be combined or otherwisere-arranged in various embodiments. For example, video player module maybe combined with another module into a single module. The memory 74 maystore a subset of the modules and data structures identified above.Furthermore, memory 74 may store additional modules and data structuresnot described above.

The device 10 may be configured so as to allow operation of a predefinedset of functions on the device be performed exclusively through a touchscreen 70 and/or a touchpad. By using a touch screen and/or a touchpadas the primary input/control device for operation of the device 10, thenumber of physical input/control devices (such as push buttons, dials,and the like) on the device 10 may be reduced.

The predefined set of functions that may be performed exclusivelythrough a touch screen and/or a touchpad may include navigation betweenuser interfaces. In some embodiments, the touchpad, when touched by theuser, navigates the device 10 to a main, home, or root menu from anyuser interface that may be displayed on the device 10.

The device 10 as shown in FIG. 3 may comprise more or fewer componentsthan shown, may combine two or more components, or a may have adifferent configuration or arrangement of the components. The variouscomponents shown in FIG. 3 may be implemented in hardware, software or acombination of both hardware and software, including one or more signalprocessing and/or application specific integrated circuits.

Components shown in FIG. 3 may communicate over one or morecommunication buses or signal lines 103.

According to some embodiments presently disclosed, the device 10comprises a motion sensor 35, orientation sensor 40, temperature sensor45, distance sensor 50, and/or a plurality of light sources 55.According to some embodiments presently disclosed, the device 10 mayalso comprise hand controls 80 and/or an illumination driver 85.

According to some embodiments, the illumination driver 85 controls andprovides suitable power to the light sources 55. The light sources 55may be activated by the illumination driver 85 in response to one ormore signals from the system processor module 65. According to someembodiments, the light sources 55 are operated in synchronization withthe optical sensor(s) (i.e. cameras) 25 and/or 30 to acquirefluorescence image data under appropriate excitation wavelengthillumination for each camera 25 and/or 30. The light sources 55 can beoperated in continuous or pulsed illumination modes. The pulse modefacilitates background image capture to enhance detectability inbrighter ambient light. The illumination driver 85 receives one or moresignals from the system processor module 65 to turn the light sources 55on and off. During fluorescence imaging modes the light sources 55 areturned on and off sequentially via one or more signals from the systemprocessor module 65.

According to some embodiments, the light sources 55, 60 may be lasers,light emitting diodes (LEDs), lamps, or other sources of illuminationcapable of providing the appropriate wavelengths for fluorescenceexcitation. According to some embodiments, the light source 55 are highpower LEDs in the wavelength range of UV and blue/violet. According tosome embodiments, the light source 55 provide illumination time forfluorescence imaging of between 1 msec to 200 msec for each excitationwavelength. The actual time of the exposure for either fluorescenceimaging may be controlled by a system software algorithm which takesinto account the task being performed, distance to the surface,illumination light energy, required energy for excitation, requiredenergy for disinfection, and other factors to calculate the illuminationand imaging times.

According to some embodiments presently disclosed, the plurality oflight source 55 may provide illumination. According to some embodiments,the plurality of light sources 55 output spectra overlay the NADH andFAD excitation maxima (440 nm and 535 nm). According to some embodimentspresently disclosed, the plurality of light sources 55 excite porphyrinfluorescence from bacterial load in the wound to allow measurement ofthe presence and extent of infection. According to some embodiments, theplurality of light sources 55 provide 725 nm and 797 nm illuminationwavelength for oxygenation analysis.

When the task being performed is fluorescence imaging the system setsthe illumination time based on the amount of energy the illuminationsystem provides under UV illumination and under blue violet illuminationat a known distance that was determined by measurement during a systemcalibration process. The system software determines the amount ofillumination required for detection of a desired contaminant, such assaliva or biological residues or bacteria, from prior knowledgeextracted from experimental measurements with known samples.

According to some embodiments, the camera 25 is configured (i.e.optimized) for collection of fluorescence image data that can indicatethe presence of specific contaminants on the biological tissue beingexamined. According to some embodiments, the camera 30 is configured(i.e. optimized) for collection of fluorescence images from organicresidues and other contaminants on the surface being examined.

According to some embodiments, the camera 25 and/or 30 are equipped withlenses optimized to pass the desired wavelengths and placed so that thefields of view of each camera 25 and/or 30 overlap such that therespective images can be subsequently processed to provide registeredimage data with a common field of view. According to some embodiments,at least one camera 25 or 30 is used in “view finder mode” to guide theoperator when aiming the cameras 25, 30 toward targeted surfaces duringinspection and disinfection. In “view finder mode”, the cameras 25 or 30is imaging under ambient light illumination or other light sourcesintegrated in the device 10.

According to some embodiments, the system processor module 65 comprisesa computer on an integrated circuit with a Central Processing Unit (CPU)with machine learning model computation, multiple data input and outputports, and peripheral device interfaces with connection to various othercomponents as shown in FIG. 3 . The system processor module 65 may hostthe system software that guides inspections, analyzes data, andcommunicates with the user (i.e. operator) of the device 10 and one ormore external servers 90, 95. The system processor module 65 may providecontrol of the cameras 25, 30 and the light sources 55 for imaging. Thesystem processor module 65 may manage the timing and synchronization ofthe light sources 55 with the capture of the fluorescence images by thecameras 25, 30. The system processor module 65 may process the capturedimages to provide meaningful information to operators and for inspectionrecords.

The system processor module 65 may be configured to communicate with andanalyze information from at least one of, the distance sensor 50, themotion sensor 35, the orientation sensor 40 and the image sensor 28, 31to determine whether conditions are unsafe to activate the light source55. The system processor module 65 may determine the appropriateexposure time for disinfection illumination, to ensure disinfection fora particular target species at a particular distance to the surface tobe disinfected, or to control and monitor the disinfection process todetermine if during disinfection there was sufficient lack of motion ofthe handheld device to ensure complete disinfection.

The system processor module 65 may be configured to communicate with andanalyze information from at least one of the distance sensor 50, themotion sensor 35, the orientation sensor 40 and the image sensor 28, 31to determine whether conditions are suitable for image capture such thatthe image is not blurred, and that sequentially captured images are ofthe same location on the surface.

According to some embodiments, the distance sensor 50 comprises at leastone Light Detection and Ranging (LIDAR) sensor mounted in aforward-facing direction of the device 10 (shown in FIG. 1 ) anddirected towards the field of view of the surface being examined.According to some embodiments, the angular acceptance of the LIDARsensor can be adjusted programmatically to overlap a desired field ofview of the camera systems. According to some embodiments, multipleLIDAR sensors can be used to overlap different portions of the fields ofview of the cameras 25, 30. This may be useful if a surface of patient'sskin is irregular or of narrow width. According to some embodiments,sensor 50 measures the depth and topology of the biological tissue.

The system processor module 65 may be configured to receive andinterpret signals from the hand actuated controls 80 of the device 10.Hand actuated controls 80 can include momentary push button switches,on/off push button switches, or multi-axis push button controls that canbe used to guide a cursor on the display 70.

According to some embodiments, the device server 90 comprises a computersystem connected either wirelessly or by a secure wire or fiberopticconnection to the device 10. According to some embodiments, the deviceserver 90 is a cloud server. The device server 90 may be configured tohost the image and inspection history databases for one or more devices10 and communicates with the system software on one or more devices 10.According to some embodiments, the device server 90 manages thecommunication of data and reports to and from one or more externalservers 95.

According to some embodiments, the one or more external servers 95 maybe customer servers or servers providing other data such as localenvironmental conditions or local disease prevalence. The device server90 may also host web portals where users of the device 10 and or theirmanagers can view inspection histories, incident reports, device status,inspection status, and where users can setup inspection task lists andperform other management and reporting functions regarding cleanlinessstatus and completeness of the tasks for an inspection task list,multiple inspection task lists for multiple handheld devices oroperators, of a facility, or of multiple facilities.

According to some embodiments presently disclosed, the device 10comprises ventilation ports 59 directing cooling airflow exhausts fromfan driven cooling system 107.

According to some embodiments presently disclosed, the system softwareis fully or partially stored in memory of the device server 90.According to some embodiments presently disclosed, the system softwareruns on the device server 90.

According to some embodiments presently disclosed, the system softwaremay provide data storage for measurements and other information that aretransferred from the device 10. The system software on the device server90 may provide system management functions for managing the creation ofjobs and task lists that can be implemented using the device 10. Thesystem software on the device server 90 may be configured to manage datastorage and creation of jobs and task lists for one or more devices 10for an organization. For example, a company may have five devices 10 atdifferent locations that are managed from a single device server 90.According to some embodiments, the device server 90 may also manage datastorage and creation of jobs and task lists for multiple organizationswith multiple devices 10.

According to some embodiments presently disclosed, the device server 90is a cloud server wirelessly connected to one or more devices 10 andproviding services to many organizations. The cloud device server 90 maycomprise web portals that are accessible through the internet whereusers or managers can manage one or more devices 10. The systemmanagement software on the device server 90 may provide for thecreation, storage, and retrieval of inspection and sanitation reports.The system management software on the device server 90 may provide forthe creation of a risk index for each inspection task and for analysisof previous inspection and sanitation reports to analyze ongoing riskand apply an updated risk index for each inspection task. The systemmanagement software on the device server 90 may provide the ability tocommunicate with external sources of data. External sources of data canbe at least one of an organization server, an institutional server, aserver providing data from a government or regulatory body, a serverproviding data from a public or private source of environmental, health,epidemiological, weather, population, scheduling, transportation, etc.information. The management software on the device server 90 may alsoprovide data to local, regional, national, or international agencies orregulatory bodies.

The device server 90 may communicate task management information andcollect data via wired or wireless methods to the system software on thedevice 10. The system software can communicate reports and measurementdata and device 10 system status to the device server 90. The systemsoftware may comprise firmware software, analysis software, and userinterface software.

The user interface software provides information and control screens onthe display 70 to guide a medical professional and/or technician (i.e.user and/or operator) through diagnosis of the biological tissue anddiagnosis task list. According to some embodiments, the user interfacesoftware displays options to the operator via the display 70 and acceptsinput from the operator via either the display 70 or the hand controls80 on the smart display and/or accepts input from the operator via thesmart display 70 and the hand controls 80 on the device. According tosome embodiments, the user interface software provides for communicationof inspection tasks, inspection status and inspection results to thedevice server 90.

The firmware software may be directly connected to and controls thehardware components of the device 10. The user interface softwareprovides information to and interprets commands from the device 10operator. The analysis software continuously analyzes sensormeasurements 35, 40, 50, 45, analyzes image data, and providesinformation to the operator to guide the diagnosis of the biologicaltissue.

The firmware software prepares the optical sensors 25, 30 and the lightsources 55 for image capture by setting appropriate parameters for eachoptical sensor including exposure time, camera gain, camera offset,pixel binning, and other programmable settings of the camera appropriatefor capturing the desired image. The firmware software also sets theon-times and off-times for each source of illumination.

According to some embodiments, presently disclosed system and methodcomprises an analytical multimode optical system that provides at leasttwo methods, including but not limited to fluorescenceimaging/spectroscopic, reflectance imaging/spectroscopic, scatteringimaging/spectroscopic, and/or Raman imaging/spectroscopic analysis of asample using at least one transmission source, such as a light source,and potentially a sensor or collection element.

According to some embodiments, presently disclosed system and methodprovides an incident light beam, for example, traveling to a patient'sskin via a wide field, or point illumination at the same region orperiphery of the collection optics. The presently disclosed system thencollects the emitted light from the patient's skin and forwards theemitted light to a photodetector and/or camera for analysis and viewing.The patient's skin can also be illuminated with a laser to generateRaman emissions, fluorescence emissions, or a scattering pattern (e.g.speckle), which are then collected through the same or different opticalpath and provided to a spectrograph or sensor for wavelengthidentification. According to some embodiments, presently disclosedsystem and method employs multiple modes of inspection, and the mode ormodes employed may depend on the type of biological tissue beinginspected. According to some embodiments, presently disclosed system andmethod employs feedback to cross-validate or verify different modes ofoperation and may in some instances be used to adjust inspectionparameters.

According to some embodiments, presently disclosed system includes aninspection tool and an associated method of inspection. For example, thepresently disclosed system may take the form of a hand-held device orpossibly small tabletop device, that incorporates multiple transmissionand/or sensing devices, such as light emitting and sensing devices,multiple corresponding spectral detection systems, and communication andanalysis devices and methods. According to some embodiments, presentlydisclosed device enhances the ability of a medical professional and/ortechnician to analyze at least a portion of patient's skin andcommunicate its compositional, molecular, and/or chemical constituents.

According to some embodiments, presently disclosed system and methoduses multimode optical imaging to greatly reduce the time required forevaluation of patient's skin and identifying the one or more aspects ofthe biological tissue. The multiple modes may include but are notlimited to the various modes presented and discussed below.

According to some embodiments, presently disclosed system and methoduses Hyperspectral Imaging (HSI) functions by integrating conventionalimaging and spectroscopy to gain spatial and spectral information frompatient's skin and/or subdermal tissue. HSI is capable of capturingreflectance, transmittance, and fluorescence images in the visible andinfrared regions with sub-millimeter spatial resolution and highspectral resolution (10 nm). Advantages HSI provides in comparison toother techniques (such as RGB imaging, NIR spectroscopy, and multi-colorimaging) include the ability to produce spatial and spectralinformation, multi-constituent information, and sensitivity to minorcomponents. HSI in the near infrared (NIR) can provide chemicalcomposition present in the biological tissue, such as prediction of fat,protein, and water content. Moreover, HSI enables the detection ofcertain bacteria that might be present in the biological tissue, suchas, for example E. coli. Fungal growth in the biological tissue is ofparticular concern due to the potential for detrimental effects onpatient's health. HSI may also be used to identify fungal species suchas Aspergillus flavus, Aspergillus parasiticus, Aspergillus niger andFusarium spp.

Another source of contamination of the biological tissue is fecalcontamination. Multispectral detection of fecal contamination using HSIimaging has been demonstrated A HSI system with a range of 450 to 851 nmmay be used to examine reflectance images for the presence of fecalcontamination. Fecal contamination sites may be evaluated usingprincipal component analysis (PCA) with the goal of identifying two tofour wavelengths that can be used in an online multispectral imagingsystem. According to some embodiments, fecal contamination can beidentified using either of three wavelengths in the green, red, and NIRregions.

One of the main advantages that HSI has over conventional spectroscopymethods is its ability to provide visual distribution maps ofcontamination in a pixel-wise manner Multiplication of regressioncoefficients of a multiple linear regression model by the spectrum ofeach pixel in the image provides a prediction map showing thedistribution of bacteria within the biological tissue. Distribution mapsof samples examined using HSI may be found in Cheng, J. H., and Sun, D.W., “Rapid quantification analysis and visualization of Escherichia coliloads in grass carp fish flesh by hyperspectral imaging method,” Foodand Bioprocess Technology, 8(5), 951-959 (2015), the entirety of whichis incorporated herein by reference. In the Cheng and Sun reference,different bacterial loads are graphically represented by colors fromblue, (representing low or no bacteria growth) to red (representing highbacteria growth).

HSI is a non-destructive tool for direct, quantitative determination ofEnterobacteriaceae loads in the biological tissue. Such a processemploys partial least squares regression (PLSR) models and root meansquared errors. Such use of HSI entails a simplified PLSR model thatpredicts Enterobacteriaceae loads in every pixel of the image acquiredfrom HSI, resulting in a new image called a ‘prediction map’ Theprediction map uses a color scale to represent describe the differentmicrobial loads in each spot of the sample.

Feng, Y. Z., ElMasry, G., Sun, D. W., Scannell, A. G., Walsh, D., andMorcy, N, “Near-infrared hyperspectral imaging and partial least squaresregression for rapid and reagentless determination of Enterobacteriaceaeon chicken fillets,” Food Chemistry, 138(2), 1829-1836 (2013), theentirety of which is incorporated herein by reference, shows an image ofa median-filled prediction map using a simplified PLSR model built onwavelengths of 930, 1121, and 1345 nm. The values under each samplerepresent predicted Enterobacteriaceae counts in log₁₀ CFU g⁻¹. As shownin the Feng et al representations, when the microbial loads increase,the images shift from a blue color to a more reddish one, reflecting thegrowth of bacteria.

According to some embodiments, presently disclosed system and methoduses Raman Spectroscopy and Spectral Imaging. Raman spectroscopy is anon-destructive spectroscopic technique, based on the vibrationalproperties of the constituent molecules, that provides molecularinformation about the sample under examination. The Raman signal resultsfrom molecules excited by a small amount of incident light at a specificwavelength. The remitted light has some photons shifted to differentwavelengths by the addition or subtraction of vibrational energy fromsome of the tissue intra-molecular bonds. Contrast is achieved when thetissue molecular constituents differ such that the Raman signals fromtwo tissues have different wavelength distributions. Raman SpectralImaging (RSI) intertwines Raman spectroscopy and digital imaging tovisualize the composition and structure of a target, which is useful inthe biological tissue analysis. Although its signal-to-noise is low,Raman imaging is a highly specific and sensitive technique that allowsfor the detection of particular chemicals at low concentrations. Onestudy aimed at the detection and differentiation of waterborne bacteria(E. coli. Staphylococcus epidermidis. Listeria monocytogenes, andEnterococcus faecalis) used surface-enhanced Raman spectroscopy (SERS)coupled with intracellular nanosilver as SERS substrates. Variationsobserved in the spectral patterns of bacterial pathogens resulted fromdifferent quantity and distribution of cellular components such asproteins, phospholipids, nucleic acids, and carbohydrates. SERS coupledwith statistical analysis is useful in discriminating and detectingbacterial cells, spores, and viruses.

According to some embodiments, presently disclosed system and methoduses Raman sensor system with an integrated 671 nm microsystem diodelaser as excitation light source for the rapid in situ detection ofbacteria. The system used in this situation demonstrates a reduction inform factor enabled by recent advances, where such a system includesthree main components: a handheld measurement head with a laser driverelectronics board, the Raman optical bench, and a battery pack. Such asystem has been employed to rapidly detect, for example, musculuslongissimus dorsi (LD) and musculus semimembranosus (SM). The totalnumber of mesophilic aerobic microorganisms exhibit possiblecorrelations of bacterial growth with the measured Raman spectra.

According to some embodiments, presently disclosed system and methoduses Speckle (scattering) Imaging. Undesirable microorganisms includebacteria, yeast, and mold. Laser Speckle imaging is presently used tomonitor moving particles in optically inhomogeneous media by analyzingtime-varying laser speckle patterns in detecting contaminants. Lightpaths associated with the movements of living microorganisms result intime-varying changes in speckle intensity patterns. By detecting thedecorrelation in the laser speckle intensity patterns from tissues, theliving activities of microorganisms can be detected Bacterial coloniescan be detected within a few seconds using Speckle imaging. Presentlydisclosed system and method provides an efficient and effective way todetect live bacteria in the biological tissue. Speckle imaging systemssense the presence of bacterial colonies and other contaminants in thebiological tissue.

Laser Speckle Contrast Imaging (LSCI) is a non-scanning opticaltechnique used in observing, for example, blood flow in medicalapplications or live bacteria colonies. Speckles are produced whencoherent light scattered back from biological tissue is diffractedthrough the limiting aperture of focusing optics. Mobile scatterers,i.e. scattering objects or items that are moving, cause the specklepattern to blur. According to some embodiments, the presently disclosedsystem design employs a model that inversely relates the degree of blur,called “speckle contrast,” to the scatterer speed. In tissue, red bloodcells are the main source of moving scatterers. Bacteria movement actsas a virtual contrast agent.

One label-free bacterial colony phenotyping technology is the BARDOT(Bacterial Rapid Detection using Optical Scattering Technology) system,which can provide classification for several different types ofbacteria. A certain speckle formation allows for the detection andidentification of these bacterial species. As the center diameter of theBacillus spp. colony grows from 500 to 900 microns, the average specklearea decreases two-fold in certain experiments and the number of smallspeckles increases seven-fold. As Bacillus colonies grow, the averagespeckle size in the scatter pattern decreases and the number of smallerspeckle increases due to the swarming growth characteristics of bacteriawithin the colony.

According to some embodiments, presently disclosed system and methoduses Fluorescence spectroscopy to identify and differentiate bacteriaFluorescence spectroscopy is a simple, non-destructive, non-invasive andrelatively inexpensive analytical method. In comparison with otherclassical analytical methods, fluorescence spectroscopy providesenhanced selectivity, high sensitivity to a wide array of potentialanalytes, and has no requirement for consumable reagents or extensivesample pre-treatment. This technique is based on the intrinsicfluorescence of bacterial cell components. When examined withultraviolet light, aromatic amino acid residues (tryptophan, tyrosine,phenylalanine), nucleic acids, and co-enzymes are intrinsicfluorophores. However, due to the multicomponent nature of items such asfoods, their fluorescence spectra are complex and chemometric methodsusing multivariate analysis are needed to extract contaminant specificinformation. Presently disclosed system and method may vary both theexcitation and detection wavelengths, and measure both reflectance andfluorescence emission properties of the biological tissue. Presentlydisclosed system and method is adjusted or may adjust for specificcontaminants. For biological tissues, dual or multiple excitationfluorescence can increase the specificity and accuracy of classificationand quantification of specific sources of fluorescence. When the systememploys dual excitation wavelengths, fluorescence emission contributionsof contaminants can be more precisely detected, and the presentlydisclosed system may disregard other irrelevant or unnecessaryfluorescence components of the biological tissue. According to someembodiments, Ratiometric versions of this approach may be employed.

According to some embodiments, presently disclosed system and methoduses reflectance and fluorescence hyperspectral imaging. Hyperspectralimaging integrates spectroscopic and imaging techniques to enable directidentification of different components and their spatial distribution inthe biological tissue. The resulting three-dimensional dataset or‘hypercube’ contains two spatial dimensions and one spectral dimension.The advantages of hyperspectral imaging over traditional methods includeno, or minimal, sample preparation, no contact, nondestructive nature,fast acquisition times, and visualization of the spatial distribution ofnumerous components simultaneously.

According to some embodiments, presently disclosed system and methodcombines several optical imaging methods, for example a combination ofreflectance and fluorescence spectroscopy together with dynamic speckleimaging. Fluorescence spectroscopy is a simple, non-destructive,non-invasive and relatively inexpensive analytical method that providesenhanced selectivity, high sensitivity to a wide array of potentialanalytes, as well as no requirement for consumable reagents or extensivesample pre-treatment. Fluorescence spectroscopy is based on theintrinsic fluorescence of bacterial cell components. When examined withultraviolet light, aromatic amino acid residues, nucleic acids, andco-enzymes are intrinsic fluorophores. However, due to themulticomponent nature of the biological tissue, their fluorescencespectra are complex, and chemometric methods using multivariate analysisare employed to extract contaminant specific information By varying boththe excitation and detection wavelengths and measuring both reflectanceand fluorescence emission properties of the biological tissue, thepresently disclosed system can be employed to accurately assess specificcontaminants. For biological tissues, dual or multiple excitationfluorescence can increase the specificity and accuracy of classificationand quantification of specific sources of fluorescence. According tosome embodiments, the presently disclosed system employing dualexcitation wavelengths allows for more specific detection offluorescence emission contributions of contaminants and disregard otherfluorescence components.

According to some embodiments, presently disclosed system and methoduses Infrared spectroscopy, for example, is a fast, sensitive, andnon-destructive technique that may be used to analyze the biologicaltissue. Analyzing the biological tissue using the mid infrared spectrum(4000-400 cm⁻¹) can give valuable information about the existence ofmolecular bonds. Such details can help determine the types of moleculespresent in the biological tissue. According to some embodiments,presently disclosed system and method employ Fourier Transform InfraredSpectroscopy (FTIR) combined with attenuated total reflectance (ATR) andpartial least square regression (PLSR) to detect the presence oflard/fat in the biological tissue.

The FTIR spectra of both mutton body fat (MBF) and lard is shown inJaswir, I., Mirghani, M. E. S, Hassan, T. H., and Said, M. Z. M.,“Determination of lard in mixture of body fats of mutton and cow byFourier transform infrared spectroscopy,” Journal of oleo science,52(12), 633-638 (2003), the entirety of which is incorporated herein byreference. One representation in Mirghani, et al. shows distinctdifferences in the raw spectra obtained between MBF and lard. Thefrequency region 3010-3000 cm⁻¹ indicates a significant differencebetween lard and MBF The lard spectrum has a sharp band at higherfrequency (3009 cm⁻¹) than MBF which has a shoulder peak at lowfrequency (3001 cm⁻¹). FTIR inspection provides a clear and concisemanner to identify lard in a mixture of other fats.

According to some embodiments, presently disclosed system and methoduses visible and near infrared reflectance spectroscopy (VIS-NIRS).VIS-NIRS combined with partial least square regression analysis can beused to analyze the biological tissue through analysis of the spectraldata collected by reflectance.

According to some embodiments, presently disclosed system and methoduses multimode hyperspectral imaging system. Due to the multicomponentnature of biological items such as the biological tissue, theirreflectance or fluorescence spectra are complex. Chemometric methodsusing multivariate analysis may be employed by the present system toextract contaminant specific information. By varying both the excitationand detection wavelengths and measuring both reflectance andfluorescence emission properties of the biological tissue, profiles maybe assessed, refined, and employed when examining specific contaminants.For biological tissues, dual or multiple excitation fluorescence canincrease the specificity and accuracy of classification andquantification of specific sources of fluorescence. The combination ofdifferent spectroscopic methods (such as fluorescence and NIRspectroscopy) circumvents single method inherent limitations and canemploy optical sensing for in situ mycotoxin detection. Additionalchemometric tools eliminate factors related to disturbing the biologicaltissue and enable extraction of desired biochemical information withrespect to contamination with fungi and/or mycotoxins.

The multimode hyperspectral imaging system may operate in fluorescenceand reflectance modes and may concurrently, or at a different time,employ speckle imaging. One example of such a system is presented inFIG. 4 . The system uses spectral band sequential imaging on thedetection side. To ensure high signal to noise level, camera andspectral selection filter integration time is optimized for eachspectral band from visible to the near infrared. The illumination moduleuses two independent light sources to provide illumination forfluorescence excitation and reflectance measurements using threecomputer-controlled LED illumination rings. The UVA (375 nm) andblue/violet (420 nm) LED rings provide fluorescence excitation. WhiteLEDs will be used for reflectance illumination. The HSi-440COHyperspectral Imaging System (Gooch & Housego, UK, originally developedby ChromoDynamics, Inc.) incorporated in the presently disclosed systemcan image and analyze multiple signals in fixed and living cells atvideo rates. Its tunable filter can switch wavelengths withinmicroseconds. The system acquires multi-wavelength, high-spatial andspectral resolution image datasets, and can compute and displayquantitative signal-specific images in near real-time. The spectrallycontrollable image capture system can record spectral images of foodsamples in wavelengths ranging from 450 nm through 800 nm. The system isconfigured as a tabletop platform where illumination and detectionoperate above the biological tissue. In this system, time-varyingspeckle signals can be quantitatively addressed with speckle correlationtime. A sample containing living microorganisms has a correlation timeshorter than a static one, and thus contaminated biological tissue isless time-correlated due to the spontaneous motility of microorganismsCorrelation time of scattered light from biological tissue, as well aspresence and activity of microorganisms are quantitatively analyzed.

Oxidative stress plays a vital role in diabetic wound healing. Animbalance of free radicals and antioxidants in the body results inoverproduction of reactive oxygen species which leads to cell and tissuedamage, and delayed wound healing. Decreasing reactive oxygen species(ROS) levels by using antioxidative systems may reduce oxidativestress-induced damage and improve healing.

According to some embodiments, presently disclosed system and methodmeasures Oxidative stress that causes cellular injury in diabetes,Tissue vascularization and oxygenation, and provides Temperaturemapping. Oxidative metabolism of glucose in mitochondria is the majordriver of oxidative stress in diabetes. FAD and NADH are autofluorescentcoenzymes of the mitochondrial respiratory chain whose levels increasewith high blood glucose and can be monitored using optical techniques.Mitochondrial redox ratio (NADH/FAD) is a measure of mitochondrialdysfunction, central to the pathophysiology of diabetic wounds. Tissuevascularization and oxygenation, is a measure of adequate blood flowwhich determines the availability of oxygen to wound tissue. Perfusionand delivery of oxygen to tissue are important in wound healing and canbe measured by spectral imaging. Temperature mapping provides anunderstanding of heat distribution when overlaid on a digital 3D modelof the wound and neighboring skin tissue. Previous studies usingtemperature mapping of skin tissue have empirically determined that adifference of 2.2° C. between corresponding limbs of the same individualis a sign of abnormal skin circulation.

Presently disclosed system and method provide sensitivity andspecificity for discriminating between diseased and non-diseased tissue,while operating non-destructively and in real-time when measuring intactorgans and in vivo. Although oxygen is a necessary component for complexorganisms that demand high energy, overproduction of ROS may cause DNAdamage, cell death, and protein modifications that result inmitochondrial and cellular dysfunction. The fluorescence signals ofintrinsic tissue fluorophores NADH and FAD provide indicators of tissuemetabolism in tissue injury due to hypoxia, ischemia, and cell death.These fluorophores are autofluorescent and can be captured without theuse of any exogenous labels at an excitation/emission (Ex/Em) wavelengthpair of 365 nm/460 nm for NADH and 440 nm/520 nm for FAD as shown inFIG. 5 . The mitochondrial redox ratio (RR=NADH/FAD) is a quantitativemarker of the mitochondrial redox state of intact tissue and in vivotissue.

According to some embodiments, presently disclosed system and methodcomprise fluorescence imaging system for measuring oxidative stress andmitochondrial redox. Referring to FIG. 6 a , NADH and FAD images werecaptured at the surface of the exposed wound in a mouse model todetermine the redox ratio. FIG. 7 a-b show 3D rendered metabolic images(NADH, FAD, and redox ratio) obtained from representative diabetic andcontrol wounds. FIG. 6 b shows the histogram comparison of thevolumetric redox ratio. The fluorescence images of the diabetic mouseshows a 66% lower volumetric redox ratio (oxidized state), lower NADH,and higher FAD fluorescence signals compared to the control. Accordingto some embodiments, NADH and FAD provide prognostic value in assessingwound healing in diabetic wounds.

According to some embodiments, presently disclosed system and methodmeasure variations in tissue oxygenation. According to some embodiments,presently disclosed system and method provide imaging analysis for skinoxygenation that is independent of skin melanin content. Melanin in theepidermis acts like a curtain in she skin, preventing some photons fromreaching the dermal layers. When measuring oxygenation, this createsmelanin and hemoglobin absorption spectra crosstalk due to theoverlapping spectral absorption characteristics of oxyhemoglobin,deoxyhemoglobin, and melanin. High melanin content can be misinterpretedas high deoxyhemoglobin content and produce errors for total hemoglobinand oxygen saturation in people with darker skin or in more pigmentedregions of the skin. Inconsistent estimations may also stem frominsufficient discrete wavelength measurements. According to someembodiments, broader spectral measurement can significantly improve theaccuracy of oxygenation calculation in subjects with deeper skin tone.

According to some embodiments, presently disclosed system and methodprovides a 3D model of the biological tissue in which the thermal image,from thermal camera, is overlaid on a 3D digital model of woundtopology. Monitoring of foot skin temperature to detect whether thetemperature difference between contralateral feet exceeds a specifiedthreshold can effectively reduce the incidence of recurrent plantarulcers.

Referring to FIG. 8 a , a setup for determining 3D maps of skin tissuewith a temperature map overlaid for a foot phantom is shown according tosome embodiments presently disclosed. FIG. 8 b shows the 3Dreconstructed image volume of foot and the thermal image on the ball ofa foot phantom. According to some embodiments presently disclosed, afoot scanning unit at the clinic or at home, provides a patient with amolecular, physiological, and visual analysis of the skin and underlyingstructures of their feet to assess tissue health and improve theirquality of life through better disease management. The presentlydisclosed system can integrate with telehealth systems and electronichealth record systems. Clinicians can receive an easy-to-understandreport that can help spot problems immediately and track healthprogress. Accurately mapping hemoglobin distribution can be useful toidentify atypical vascular patterns and issues related to melanin depth(i.e. melanoma).

Increased oxidative stress is a major contributor to diabeticcomplications, including retinopathy, nephropathy, and cardiovasculardisease. Oxidative stress also plays a significant role in regulatingwound healing and the resulting redox imbalance has major implicationsin diabetic wounds. Coenzymes NADH and FAD activate mitochondrialmetabolism and mitochondrial bioenergetics are altered in dysregulatedinflammatory processes observed in the pathophysiology of chronicnon-healing wounds. Moreover, optical redox ratio maps produced fromwound examinations are associated with a transient increase inkeratinocyte proliferation at the wound edge. Cutaneous tissueoxygenation also plays a role in wound healing in diabetic patients.Chronic or prolonged peri-wound and wound bed hypoxia delays healingbecause tissue metabolic demand increases during recovery from tissueinjury while oxygen delivery and availability are reduced. Multimodeoptical imaging can quantify tissue oxygenation at a microvascular scaleand produce a map of microcirculatory changes reflecting oxyhemoglobinand deoxyhemoglobin concentration. Mean oxygenation values in theperi-wound area show significant differences between diabetic ulcersthat healed and those that did not. Table 1 identifies, for example, atleast five biomarkers that are predictive of wound healing in diabeticfoot ulcers.

TABLE 1 Wound biomarkers Fluorescence Excitation/ Biomarker EmissionReflectance Diagnostic Marker NADH 365 nm 440 nm Metabolic State FAD 436nm 520 nm Porphyrin 415 nm 630/ Infection Oxygenation 690 nm 725 nmTissue Oxygenation Oxygenation 797 nm Tissue Oxygenation Vascularture520 nm Vascularization Temperature Wound temperature

In clinical practice, evaluation of wound status is based on wound size,odor, drainage, and eschar. These parameters are obtained by directsurface measurement and visual observation. Those gold standards ofgross wound evaluation, however, cannot provide functional or structuralinformation from below the wound surface and can be very subjective.According to some embodiments, presently disclosed multimode opticalimaging provides quantitative biomarkers of wound state with functionalor structural information.

The gold-standard in ulcer assessment is visual inspection plus swabbingfor microbiology or biopsy to perform histological analysis. Biopsy isinvasive and could potentially introduce a new wound that complicatesthe wound healing process and the precision of assessment of the woundor ulcer over time. However, presently disclosed optical imaging device10 provide a quantitative and noninvasive imaging tool that can be usedby the clinician (i.e. operator) to assess the wound objectively andprovide appropriate treatment.

According to some embodiments, presently disclosed system and methoduses a plurality of optical sensors 25, 30, LED illumination and 3Dimage processing and statistical analysis to produce a high-quality,compact, and low-cost device 10 which can be operated in bright ambientlight.

According to some embodiments, presently disclosed system and methodextract four parameters—tissue metabolism, oxygenation, bacterial load,and vascular networks—and display these as wound image map overlays.According to some embodiments, presently disclosed system and methodprovides vasculature, tissue oxygenation, metabolism, and temperatureimages that cannot be seen with the naked eye

According to some embodiments, presently disclosed system and methodArtificial Intelligence (AI) and machine learning (ML) to quantitativelymeasure DFU volume, ischemia, and infection load for wound assessment.According to some embodiments, presently disclosed system and methodprovides the 3D visualization of the extremity and diabetic wound itselfas well as efficient detection of infection location and possibleischemia. According to some embodiments, presently disclosed device 10captures red-green-blue (RGB) conventional reflectance and twofluorescence images. According to some embodiments, presently discloseddevice 10 uses two wavelengths, UVA (365 nm) and violet (405 nm) lights,for excitation to create the fluorescence map or special pattern oftissue and indicate low bacterial burden (≥103 CFU/g) under ambientlight. The presently disclosed device 10 contains optical filters whichpass wavelengths related to NADH, FAD, pyoverdines, porphyrins, andconnective tissue contents. It overlays both fluorescence images on theareas with infection and automatically discriminates bacterial gram-typeusing ML to ensure accurate treatment decisions. Gram-positive bacteriafluoresce red and gram-negative ones are displayed in green based ondifferences in fluorescence intensity. Connective tissue and Pseudomonasaeruginosa emit green and blue-green (cyan) fluorescence respectively.Some color signals of tissue cannot be discriminated from bacterialinfection. By comparing the reflectance and fluorescence wound images,the clinician can overcome misinterpretation of data. Ischemia can bedetermined using conventional reflectance images. For further bacterialanalysis to indicate specific species, swabbing and tissue sampling canbe performed by the device's guidance. The real-time data lead totargeted treatment plans, including real-time monitoring of the efficacyof debridement and cleansing, and usage of appropriate antimicrobialdressings by home health nurses. After a comprehensive assessment of DFUusing presently disclosed device 10, any severe infectious or ischemicDFU images/documents may be simultaneously synced to the hospitals'database and referred to a podiatrist and vascular specialist for timelyrevascularization and other interventions. Presently disclosed systemmay implement the Pandemic Diabetic Foot Triage System to make timelyreferrals by sharing the elderly wound assessment data with a convenientsite for care. It assists DFU re-staging following debridement orrevascularization to re-evaluate the treatment plan over time, initiallyafter 4-6 weeks followed by 3-6-month intervals or even longer.Presently disclosed system may monitor wound healing by comparing thepatient's new data with the previously recorded document.

According to some embodiments, presently disclosed system and methodcharacterizes materials, typically biological tissue based on multimodespectral analysis. According to presently disclosed system and method,such analysis includes identifying features from different modes ofmeasurement. Feature extraction/selection strategy or methods fordifferent modes of measurement may differ based on measurement physicsand biological/chemical characteristics. Examples of feature extractionmethods include wavelet transform, startistical features, haralicktextural features, fractal analysis, and curvelet transform. Thepresently disclosed system and method may employ feature selectionmethods such as principal component analysis (PCA), independentcomponent analysis (ICA), curvature and/or manifold learning.

Wavelet transform is a mathematical transform to extract informationfrom a signal or an image. In one dimensional wavelet transform, theinput signal is represented at different scales called coarse and detailcomponents using a set of basic functions originally from a functioncalled mother function.

The presently disclosed system and method may, online or offline,include identifying which spectral measurement mode (or combination)will have the highest impact resulting among the top combinations.Optimization is based on a cost function (sensitivity, specificity, areaunder the curve) from a receiver operating characteristics (ROC) curve.Depending on technology complexity, the system employs desired practicalmodes.

Based on the practical modes determined, the present design links thebiology and chemical components of the biological tissue and correlatesthem with highest differentiating spectral features. Differentbiological tissues may exhibit differences in this regard. The presentlydisclosed system may run independent measurements using metabolic and/orchemical analysis of samples to validate the biological/chemicaldifferentiation between samples to determine optimal modalities. Thepresently disclosed system may use these optimal modalities to conduct apilot study with sample size greater than or equal to a number ofsamples, such as 100, sufficient for collection and analysis of enoughdata from a variety of samples in view of other parameters.

FIG. 9 illustrates the proposed architecture and high-level steps. Level1 includes Signal Conditioning, Feature extraction and Featureselection. Level 2 includes all trained models for differentapplications and classes. Level 3 fuses internal decisions (AI scores)coming from all internal AI models to get a final AI score Level 4applies a receiver operating characteristic (ROC), representing thediagnostic ability of the Classifiers as the discrimination threshold isvaried, to enforce desired balance between specificity and sensitivityof the AI system.

Processing of this type may be divided into two modules, featureextraction and classification. The feature extraction module processesthe raw data into a low dimensional feature vector that is relativelyinvariant to distortions and artifacts and is high in informationcontent, making it suitable to be used by the Classifier stage. Priorknowledge about the data and the experience acquired on building similarsystems may be employed. This stage of machine learning (ML) may requireexperimentation and fine-tuning by hand. The Classifier is usuallychosen from the large number of available generic modules and is trainedusing available data.

The presently disclosed system may employ a numerical computingframework, such as, for example, MATLAB, for model development andvalidation. The observations (input data) are raw measurements obtainedby the system Training set class “labels” may be provided by DNAanalysis. In the first stage, internal Classifiers are trainedseparately. A final classifier is obtained by fusing the prediction ofseveral internal classifiers (models).

The presently disclosed system may employ spectral acquisitions (rawsignals) where raw data includes fluorescence and reflectance spectraldata and correlative biological tissue results. FIG. 4 illustratesreflectance results while FIG. 10 shows fluorescence results,representing typical raw data signals. Biological tissues are imaged onthe left, with acquired spectral signatures shown ion the right Data maybe collected from any number of cases and used for classificationpipeline model selection and validation A holdout set may be employedfor Classifier final testing.

The system performs signal conditioning to reduce data variabilitycaused by differences in hardware probes and operating conditions. Thisincludes probe-specific calibration, dark current removal, andwavelength alignment to a unique set of wavelengths via interpolation.In addition, the system uses signal-to-noise, signal validation andsaturation tests to reject bad data samples.

Data may be initially calibrated prior to feeding to the classificationpipeline, based on individual spectroscopic data acquisition systemcharacteristics. The extraneous parts of the signal may be truncatedfrom the reflectance and fluorescence signals Valid wavelength rangesmay be obtained by examining raw data, automatically or manually, orperforming an optimization and exhaustive search to find validwavelength ranges.

The presently disclosed system may perform wavelength alignment usinginterpolation. Because the spectrometer cannot be calibrated such thatthe response is measured exactly at the same wavelengths for all units,the system may employ a reference wavelength grid to compare collectedsignals. The presently disclosed system may obtain the signal aligned tothe reference grid from the raw signal using a cubic or linear splineinterpolation from values measured by a spectrometer, and these valuesmay be used by the pipeline.

After initial signal conditioning, the system may extract features fromthe raw data. Processing of conditioned data into a low dimensionalfeature vector creates features that are relatively invariant todistortions and artifacts and valuable informational content. Thiscombination makes the results of this stage suitable to be used in theClassifier stage.

The presently disclosed system, or those providing functionality for thepresently disclosed system, may use prior knowledge about the dataduring this stage to determine optimal methods of feature extraction.These include original raw data in linear or log space, wavelettransform, statistical features, and textural features. The presentlydisclosed system may use feature level fusion by combining into a singlevector feature vector The presently disclosed system may determine orprovide a separate AI model for each feature type and will fuse outcomesof each AI in decision levels as well.

The presently disclosed system may perform feature selection usingmethods such as Principal components analysis (PCA) and IndependentComponent Analysis (ICA) algorithms for dimensionality reduction andremoving redundant features and information.

PCA is a statistical method that converts a set of observations andsensor data with some level of redundancy and correlation into a set ofuncorrelated components called principal components by use of anorthogonal transformation.

Independent component analysis (ICA) decomposes a multivariate signalinto statistical independent non-Gaussian components. ICA could be usedfor feature selection and reduction. We stack our raw data vectors in amatrix where each row is an observation ICA reduces the number ofcolumns or rearranges the information in the raw data into a smallernumber of features.

The presently disclosed system may normalize features using methodsincluding but not limited to z-score area under the curve (AUC).

For low level (internal) algorithms, a number of models may be employed:Deep learning techniques including conventional neural network (CNN) andtensor flow, Artificial Neural Networks; Support Vector Machines (SVM)including linear, non-linear; AdaBoost.

All of these machine learning methods are supervised binaryclassification models. A binary classifier is a numerical pipeline whichhas as input a numerical vector and outputs a binary decision, assigningthe input membership to one of two classes.

A deep learning model continually analyzes data with a logic structuresimilar to how a human would draw conclusions. To achieve this, deeplearning uses a layered structure of algorithms called an artificialneural network (ANN). The design of an ANN is inspired by the biologicalneural network of the human brain. This makes for machine intelligencethat's far more capable than that of standard machine learning models.

Differences between classical machine learning and AI versus deeplearning include: machine learning uses algorithms to parse data, learnfrom that data, and make informed decisions based on what has beenlearned; deep learning structures algorithms in layers to create an“artificial neural network” that can learn and make intelligentdecisions on its own; and deep learning is a subfield of machinelearning. While both fall under the broad category of artificialintelligence, deep learning powers the most human-like artificialintelligence.

The presently disclosed system may partition the data, withapproximately 70% used for training and 30% for testing. The process maybe repeated multiple times to test if data is independent andidentically distributed and is exchangeable. Such processing can help toevaluate intAI (internal AI) performance by calculating mean andconfidence interval for intAI performance. FIG. 11 shows one intAIarchitecture with feature extraction and selection (Extraction) prior toexecuting intAI.

The top level in the AI architecture of FIG. 11 is “fusion.” At thislevel, the system fuses internal (low-level) algorithms results(decisions) to obtain a final decision. With selected Classifiers, thesystem computes a weighted sum of Classifier decisions/score. The systemmakes a global decision by comparing this sum to a threshold.

From FIG. 11 , there is provided a fluorescence path and a reflectancepath, where fluorescence passes fluorescence image readings to an inputdata conditioning element, which in turn passes data to featureextraction and selection modules. Reflectance passes reflectance imagereadings to signal conditioning modules to align data usinginterpolation grid and filter invalid data using data quality filterssuch as SNR (signal-to-noise ratio). If features level fusion offluorescence and reflectance data is desired, a similar data processingchain could be applied to reflectance data as well. The features coiningfrom fluorescence and reflectance data are fused. After normalizationthe fused features are fed to a machine learning model.

According to some embodiments, the fusion of fluorescence image readingsand reflectance image readings shown in FIG. 11 may be used to: identifytissue type of the biological tissue, provide depth of the wound,provide level of penetration, provide inflammation information, providewould healing biomarkers. An alternate representation of the processingis presented in FIG. 12 .

With respect to AI training and internal Models Selection Process, thesystem may perform an exhaustive search and optimization to find bestinternal and fusion models. For the pipeline, the presently disclosedsystem may start with a machine learning model such as is outlinedabove. The presently disclosed system may then optimize the parametersof the model, seeking to maximize performance. With respect to an intAImodel selection, the presently disclosed system may perform selection ofthe best classification pipeline model using an exhaustive searchprocess over the possible combinations of algorithms and controlparameters for each stage. At each point in the exhaustive searchevaluation, the presently disclosed system may apply a data partitioningto use a portion of data for optimization and the remaining forcross-validation test. The main performance selection criterion is theaverage sensitivity/specificity for all the cross-validation tests. Thepresently disclosed system may perform deeper analysis on Classifiers,which may be individual software or hardware components or modules orcombinations thereof, that have passed a performance threshold (e.g.,find an operating point on the ROC with at least 95% sensitivity andmaximum specificity among other Classifiers).

The final model is fine-tuned using the training set for bestperformance. After data partitioning, a portion of data for AI trainingand the remaining for validation of trained AI models. The parameters ofeach stage may be fine-tuned around the values selected in the previousstep. The system may run this process multiple times to select the modelwith the best performance. The architecture of the models is the samebut the control parameters for each stage are data-driven and determinedby partitioning of the training data set. The presently disclosed systemmay validate the final model using the hold-out data set.

The presently disclosed system may employ multimode settings tocross-validate each mode of measurement. For instance, the presentlydisclosed system may acquire pure fluorescence measurements independentof light absorption (color) by reflectance and fluorescence inconcurrent measurements By analyzing biological tissue, for example,using multimode methods, the presently disclosed system may, moreaccurately, differentiate the target of interest, and may analyzesubstantially more information, thus addressing a wider range ofcharacteristics and drawing deeper and more discrete conclusions (i.e.more targeted and valuable signatures). The AI algorithm may trainitself over time to be more efficient, where more efficient means higheraccuracy and faster assessment.

One example of multimode spectral measurement involves the measurementof pure fluorescence spectra independent of light absorption. Naturalfluorescence in food samples can be excited in multiple wavelengthranges. Examples include 278 nm (targeting Vitamin B2, tyrosine, andtryptophan), 305 nm (Targeting Vitamin B6, Vitamin F, and ATP), 365 nm(NADH, Vitamin A), 395 nm (hematoporphyrin), and 405 nm (chlorophyll).However, individual food sample may absorb light differently atexcitation or emission wavelengths. By independently characterizingreflectance spectra, the system can minimize the absorption contributionto the fluorescence spectra and purify fluorescence spectral signatures.

A second example of multimode operation (multi excitation fluorescence)is to more effectively unmix the fluorophores contributions of the foodsample. Usually, emission spectra of natural fluorophores are broad andoverlap to other natural fluorophores. Multiple excitation wavelengthshelp to differentiate individual fluorophores. Some of the moleculeshave specific absorption characteristics that can be individuallycalculated and to be used to improve fluorescence unmixing progress.Thus, according to the present design, multimode may include using asingle technique in varying ways, such as at different frequencies orwavelengths.

Machine Learning operation of the present system can identify whichspectral features are important to differentiate between biologicalsamples such as food samples. Such machine learning helps the Classifierto weight specific molecular, compositional components relative to eachother for final classification. Machine learning also trains the expertsystem which combination of compositional, molecular, or chemicalcomponents becomes relevant and potentially important for classificationoptimization. Artificial intelligence in the system can establish astrategy where the classification can be optimized for either speedand/or accuracy by filtering most differentiating spectral features andremoving the redundant data.

A general overview of the present design is thus presented in FIG. 13 .At point 1001, the system determines the biological tissue forevaluation. This may be established by users or automatically, such asby a user being offered options and selecting those applicable. At point1002, the system determines, using the components offered and/or othermeans, attributes of samples for examination and modes of examination.For example, a sample may call for examination using Raman spectroscopyand infrared scanning, either by experience or based on previousobservations. In some instances, the samples be examined may have noknown best mode used, and thus experimentation may be required todetermine the desired use of mode X on sample Y Point 1003 is optional,wherein the system examines samples to determine sample profiles andprofile attributes of interest. For example, a sample with a particularknown contaminant may be examined using speckle imaging and it may bedetermined that when examining at a particular wavelength, the presenceof the contaminant becomes particularly pronounced, and thus all samplesmay be examined using speckle imaging at the particular wavelength todetermine the presence of the contaminant Alternately, if a biologicalsample such as a plant from a particular location exhibits an attributeunder infrared imaging, similar plant samples may be examined usingsimilar infrared imaging.

Point 1004 calls for identifying best modes and ensuring the best modesare available in the design. The various modes may be employed, butother modes may be provided as suggested. In some instances, examinationin a single mode at various frequencies, wavelengths, or othermeasurement quantities may be employed. Such modes and examinationattributes may be offered according to an examination and analysisprotocol. If it is determined that samples of interest must be examinedat wavelength P in mode Q, mode Q must be offered and must be able tooperate at wavelength P. Point 1005 represents generally the initiationof production, i.e. the examination of multiple samples according to thepresent design, wherein the device and modes are calibrated. Point 1006calls for examining samples using the device in the desired modes usingthe desired attributes, or in other words, according to the examinationand analysis protocol.

At point 1007, the system processes results, including makingassessments as to presence or absence of attributes, authenticationprobabilities, and so forth. Such processing employs the artificialintelligence and machine learning described herein. At point 1008, thesystem may evaluate and assess results, again using known attributes,machine learning, artificial intelligence, and/or other techniques.Results from this step importantly are fed back to point 1004,conceptually representing decisions to alter the examination andanalysis protocol as well as the mode or modes employed. As an example,the system may process thousands of samples of beef using a givenprotocol, such as examining using reflectance multiwavelength imaging atthree different wavelengths. However, examination at these wavelengthsmay offer limited results, such as a failure to determine the cut ofbeef being examined. In other words, results provided may beinconclusive. As a result, the system may augment the protocol andexamination by adding a different mode or may add a wavelength to thethree wavelengths used for examination. Thus, the present system employsfeedback of determined results to improve the overall protocol and theoverall examination and analysis process, and the protocol establishedmay be dynamically changed depending on circumstances encountered.

FIG. 12 is a general overview of the processing modules employed. Moreor different modules may be employed. From FIG. 12 , processor 1101controls all processing, including applicable machine learning,artificial intelligence, and the like. Point 1102 represents readingstaken, which are received by the processing arrangement 1103. Processingarrangement 1103 includes feature extraction module 1104 and classifiermodule 1105. The readings are received and distributed to the variousinternal AI training models 1106 a through 1106 m which generallyidentify known aspects and attributes from the readings based onexperience and/or prior readings.

A single internal AI training model may be employed or offered. Fusionmodule 1107 fuses the results from the various internal AI trainingmodels 1106 a through 1106 m.

Classifiers 1108 a through 1108 n classify the fused results asdescribed above, and overall results are provided at point 1109.Processor 1101 may then operate to provide the feedback shown in FIG. 10, determining that different modes and/or different assessments may beemployed or different attributes examined, for example.

Thus according to one embodiment, there is provided a biological sampleinspection apparatus, comprising an illumination hardware arrangementcomprising transmission and sensing hardware, the illumination hardwarearrangement configured to inspect a biological sample using at least twomodes from a group comprising a fluorescence imaging mode, a reflectanceimaging mode, a scattering imaging mode, and a Raman imaging mode, andprocessing hardware configured to operate the illumination hardwarearrangement according to a protocol comprising inspection settings ofthe at least two modes, wherein the processing hardware receives scanresults from the illumination hardware arrangement and identifiesattributes of the biological sample. The processing hardware isconfigured to employ the attributes of at least one biological sample toalter the protocol.

According to a further embodiment of the present design, there isprovided a method for inspecting at least one biological sample,comprising determining a plurality of inspection modes for inspectingthe at least one biological sample using a multimode inspectionapparatus, determining an inspection protocol for inspecting the atleast one biological sample, wherein the inspection protocol comprisesinspection settings for the plurality of inspection modes, inspectingthe at least one biological sample using the multimode inspectionapparatus according to the protocol, and altering the protocol based oninspection results for multiple biological samples.

According to another embodiment of the present design, there is provideda biological sample inspection apparatus configured to inspect abiological sample for issues, comprising illumination hardwarecomprising transmission and sensing hardware configured to illuminateand sense attributes of the biological sample, the illumination hardwareconfigured to inspect the biological sample using multiple inspectionconfigurations from at least one of a fluorescence imaging mode, areflectance imaging mode, a scattering imaging mode, and a Raman imagingmode, and processing hardware configured to operate the illuminationhardware according to a protocol comprising inspection settings for themultiple inspection configurations, wherein the processing hardwarereceives scan results from the illumination hardware and identifiesattributes of the biological sample. The processing hardware isconfigured to employ the attributes of at least one biological sampleand alter the protocol based on the attributes of the one biologicalsample.

Referring to FIG. 14 , a method 1700 is shown according to someembodiments presently disclosed. At 1710, operator turns on device 10.At 1715, operator logs into a presently disclosed system running on thedevice 10. At 1720, operator enters patient's information or searchesexternal server 90 and/or 95 to find patient's information. At 1725,operator uses one of the optical sensors of the device 10 to perform 3dimensional (3D) scan an area of patient's skin affected by wound and/orburn and/or peripheral regions. At 1730, system running on device 10identifies body landmarks. At 1735, operator uses another optical sensorof the device 10 to perform fluorescence imaging of the area ofpatient's skin affected by wound and/or burn and/or peripheral regions.At 1740, device 10 overlays fluorescence imaging (i.e. map) obtained at1735 on the 3D model scanned at 1725 using landmarks identified at 1730.At 1745, operator uses a third optical sensor of the device 10 toperform reflectance imaging of the area of patient's skin affected bywound and/or burn and/or peripheral regions. At 1750, device 10 overlaysreflectance image (i.e. map) obtained at 1745 on the 3D model scanned at1725 using landmarks identified at 1730. At 1755, operator uses thermalcameral of the device 10 to perform thermal imaging of the area ofpatient's skin affected by wound and/or burn and/or peripheral regions.At 1760, device 10 overlays thermal imaging (i.e. temperature map)obtained at 1755 on the 3D model scanned at 1725 using landmarksidentified at 1730. At 1765, device 10 measures size and/or depth of thearea of patient's skin affected by wound and/or burn. At 1770, device 10identifies tissue type (i.e. skin, fat, muscle, bone, etc for each pixelon reflectance imaging, fluorescence imaging, and/or thermal imaging. At1775, device 10 identifies and maps wound and/or burn stage based on thetissue molecular biomarkers, tissue type, wound and/or burn size, and/orsound depth. At 1780, device 10 displays results on the display 70 oranother external computer display suing, for example, Wi-Fi or othervideo communication methods. The image results may be displayed ondifferent panels, or overlays on 3D model. The operator may use imageresults to assess the wound and/or burn for more accurate treatmentdecision. The operator may use device 10 to transfer image results toserver 90. The image results may also be shared with another clinicianfor further analysis.

Referring to FIG. 15 , a method 1800 is shown according to someembodiments presently disclosed. At 1802, operator turns on and logsinto a presently disclosed system running on the device 10. At 1804,operator enters patient's information (i.e. patient's data, and/orpatient's record). Patient's information or portion of patient'sinformation may be located in memory 74 of the device 10, a memory ofthe server 95 and/or a memory of the server 90. Patient's informationmay include patient's medical record(s) and/or patient's personalinformation. According to some embodiments, presently disclosed systemor portion of the presently disclosed system may be stored in memory 74of the device 10, a memory of the server 95 and/or a memory of theserver 90. According to some embodiments, presently disclosed system orportion of the presently disclosed system may be processed by theprocessor modules 65 of the device 10, a processor of the server 95and/or a processor of the server 90.

At 1806, patient's information may be viewed on the display 70 of thedevice 10 or on another external screen According to some embodiments,patient's information includes past tests performed by medicalprofessionals, treatments performed and completed by medicalprofessional, treatments that were started and not completed by medicalprofessionals, and/or treatments that were not started by medicalprofessionals. At 1808, operator may make changes to patient'sinformation, updated patient's information, add new information (i.e.data) to the patient's record.

At 1810, operator may perform additional test(s) using the device 10.The additional test(s) to be performed may be listed in the patient'srecord viewed at 1806. The operator points device 10 towards patient'swound and or burn to capture a plurality of images. The device 10 may beactivated to capture the plurality of images when the operator activatesone or more controls 80. At 1812, the presently disclosed systemdetermines if there is sufficient dynamic range (i.e. enough light) forthe device 10 to obtain the plurality of images.

At 1814, the presently disclosed system scans patient's wound and orburn from 0 degrees to 360 degrees relative to the patient. At 1816, thepresently disclosed system determines the distance the device 10 is fromthe patient's wound and or burn. Device 10 may use distance sensor 50 todetermine the distance between the device 10 and the patient's wound andor burn.

At 1818, the presently disclosed system removes and/or compensatesagainst any jitter and/or shaking that may be caused by the operatorwhen using the device 10. This may be performed using one or moreaccelerometers 168, orientation sensor 40, one or more gyros, and/ormotion sensor 35 within the device 10.

At 1820, the presently disclosed system may determine if there is enoughdata from the plurality of pictures to move to the next step. If therewas not enough light, or the distance between the device 10 and thepatient's wound and or burn was too far/close, or if there was too muchshaking by the operator, the presently disclosed system may take theoperator to 1810 to retake the pictures. At 1821, the presentlydisclosed system may provide a report to the operator showing why thereis not enough data from the plurality of pictures. For example, at 1821,the presently disclosed system may notify the operator if the device 10or any portion of the device 10 need to be recalibrated, notify theoperator to bring the device 10 closer to the patient's wound and orburn, notify the operator to bring the device 10 further away from thepatient's wound and or burn, notify the operator to keep the device 10steady and/or to use a tripod to keep the device 10 steady, and/or maynotify the operator if there is not enough power and the device 10 needsto be recharged.

According to some embodiments, the device 10 comprise one or more 3Dstereoscopic imaging camera(s). Stereoscopic imaging camera comprisestwo sensors, spaced a first distance apart. Each sensor may operate atdifferent wavelength. The stereoscopic imaging camera takes two imagesfrom the two sensors and compares them. Since the distance between thesensors is known, these comparisons provide morphological (i.e. depth)information. The 3D stereoscopic imaging camera may generate RGB (i.e.truecolor) and/or Infrared (IR) images. According to some embodiments,the device 10 may use thermal imaging camera 502 to provide IR imagesand/or thermal images.

If the presently disclosed system determines there is enough data in thepictures taken by the 3D stereoscopic imaging camera(s), at 1822, thepresently disclosed system may perform 3D stereo vision analysis of thepictures taken at 1810 to generate (i.e. obtain) morphologicalinformation of the patient's wound and or burn. By comparing informationabout patient's wound and or burn from two vantage points, 3Dinformation can be extracted by examining the relative positions ofobjects (i.e. landmarks) in the images. The two sensors of the 3Dstereoscopic imaging camera, displaced the first distance from oneanother may be used to obtain two differing views on the patient's woundand or burn. By comparing images from the two sensors, the relativemorphological information can be obtained in the form of a disparitymap, which encodes the difference in coordinates (which could bevertical or horizontal) of corresponding image points. The values inthis disparity map are inversely proportional to the scene depth at thecorresponding pixel location.

At 1824, the presently disclosed system may perform 3D register FastPoint Feature Histograms (FPFH) and/or Random sample consensus (RANSAC)analysis of the pictures taken at 1810 to generate (i.e. obtain) ahistogram of the patient's wound and/or burn. For example, the histogrammay show different features of the light, color information, map of thescene of colors and possible image values associated with it. RandomSample Consensus (RANSAC) is an iterative method to estimate parametersof a mathematical model from a set of observed data that containsoutliers. The FPFH-RANSAC (“Fast Point Feature Histograms (FPFH) for 3DRegistration”) in combination with odometry. Odometry is the use of datafrom motion sensors to estimate change in position over time.

At 1826, the presently disclosed system may perform filter andconvolution analysis of the pictures taken at 1810 to filter and/or maskimages in the pictures to, for example, determine (i.e. locate) edges ofthe patient's wound and or burn.

At 1828, the presently disclosed system may perform ortho mapping and/orstereo mapping of the pictures taken at 1810. Ortho Mapping is a processthat corrects for geometric distortions inherent in remotely sensedimagery to produce ortho imagery products, raw images, and/or Orthoimagery products. Stereo mapping is a stereoscopy-enabled map thatprovides stereo vision through a stereo model, which is composed of twoimages of the same geographic area taken from two locations. Binocularstereo vision facilitates better image interpretation than singular monovision and depth resolution.

At 1830, the presently disclosed system may perform image mensurationanalysis of the pictures taken at 1810. Image mensuration appliesgeometric rules to determine distance, area of a 2-dimensional or3-dimensional surfaces using the information obtained from lines andangles. It may also includes measuring the height and absolute locationof a feature. According to some embodiments, image mensuration analysismay be used to identify landmarks on patient's wound and or burn. Theselandmarks may be used to overlap, overlay, and/or align differentimages.

At 1832, the presently disclosed system may perform perspectivetransformation analysis of the pictures taken at 1810. PerspectiveTransformation (Homography) may be used to change the perspective of agiven image or video for getting better insights into the requiredinformation. In Perspective Transformation, one or more points areselected on the image from which to gather information by changing theperspective. Perspective transformation may also be used to remove imagedistortion.

At 1834, the presently disclosed system may perform image processingand/or segmentation analysis of the pictures taken at 1810 to analyzeimages pixel by pixel for variety of features.

Referring to FIG. 16 , a method 1900 is shown according to someembodiments presently disclosed. The method 1900 may be used to detectinfection located on the surface wound/burn, located on peri-wound,located subcutaneous and located inside the wound region. According tosome embodiments, the method 1900 uses multi-modal fluorescencetechniques to image, filter and process infection.

The operator turns on and logs into a presently disclosed system runningon the device 10. The operator enters patient's information (i.e.patient's data, and/or patient's record). Patient's information orportion of patient's information may be located in memory 74 of thedevice 10, a memory of the server 95 and/or a memory of the server 90.Patient's information may include patient's medical record(s) and/orpatient's personal information. According to some embodiments, presentlydisclosed system or portion of the presently disclosed system may bestored in memory 74 of the device 10, a memory of the server 95 and/or amemory of the server 90. According to some embodiments, presentlydisclosed system or portion of the presently disclosed system may beprocessed by the processor modules 65 of the device 10, a processor ofthe server and/or a processor of the server 90.

The patient's information may be viewed on the display 70 of the device10 or on another external screen. According to some embodiments,patient's information includes past tests performed by medicalprofessionals, treatments performed and completed by medicalprofessional, treatments that were started and not completed by medicalprofessionals, and/or treatments that were not started by medicalprofessionals. The operator may make changes to patient's information,updated patient's information, add new information (i.e. data) to thepatient's record.

At 1902, the presently disclosed system may perform hardware calibrationcheck to confirm the hardware and/or software are calibrated and deice10 functions properly.

At 1910, operator may perform fluorescence test(s) using the device 10.The fluorescence test(s) to be performed may be listed in the patient'srecord. The operator points device 10 towards patient's wound and orburn to capture a plurality of images. The device 10 may be activated tocapture the plurality of images when the operator activates one or morecontrols 80. At 1912, the presently disclosed system determines if thereis sufficient dynamic range (i.e. enough light) for the device 10 toobtain the plurality of images.

At 1914, the presently disclosed system scans patient's wound and orburn from 0 degrees to 360 degrees relative to the patient. At 1916, thepresently disclosed system determines the distance the device 10 is fromthe patient's wound and or burn. Device 10 may use distance sensor 50 todetermine the distance between the device 10 and the patient's wound andor burn

At 1918, the presently disclosed system removes and/or compensatesagainst any jitter and/or shaking that may be caused by the operatorwhen using the device 10. This may be performed using one or moreaccelerometers 168, orientation sensor 40, one or more gyros, and/ormotion sensor 35 within the device 10.

At 1920, the presently disclosed system may determine if there is enoughdata from the plurality of pictures to move to the next step. If therewas not enough light, or the distance between the device 10 and thepatient's wound and or burn was too far/close, or if there was too muchshaking by the operator, the presently disclosed system may take theoperator to 1910 to retake the pictures. At 1921, the presentlydisclosed system may provide a report to the operator showing why thereis not enough data from the plurality of pictures. For example, at 1921,the presently disclosed system may notify the operator if the device 10or any portion of the device 10 need to be recalibrated, notify theoperator to bring the device 10 closer to the patient's wound and orburn, notify the operator to bring the device 10 further away from thepatient's wound and or burn, notify the operator to keep the device 10steady and/or to use a tripod to keep the device 10 steady, and/or maynotify the operator if there is not enough power and the device 10 needsto be recharged.

The device 10 comprises multimodal light sources 55 that operate atleast at two different wavelength 1903 and 1905. According to someembodiments, taking plurality of images (i.e. pictures) while flashinglight sources 55 at different wavelengths 1903 and 1905 allows presentlydisclosed system to excite, detect and identify infection present in andaround patient's wound and/or burn.

At 1922, the presently disclosed system may perform 3D fluorevisionanalysis of the pictures taken at 1910 to detect and identify infectionpresent in and around patient's wound and/or burn.

At 1924, the presently disclosed system may perform 3D register FastPoint Feature Histograms (FPFH) and/or Random sample consensus (RANSAC)analysis of the pictures taken at 1910 to generate (i.e. obtain) ahistogram of the patient's wound and/or burn. For example, the histogrammay show different features of the light, color information, map of thescene of colors and possible image values associated with it. RandomSample Consensus (RANSAC) is an iterative method to estimate parametersof a mathematical model from a set of observed data that containsoutliers. The FPFH-RANSAC (“Fast Point Feature Histograms (FPFH) for 3DRegistration”) in combination with odometry. Odometry is the use of datafrom motion sensors to estimate change in position over time.

At 1926, the presently disclosed system may perform filter andconvolution analysis of the pictures taken at 1910 to filter and/or maskimages in the pictures to, for example, determine (i.e. locate) edges ofthe patient's wound and or burn.

At 1928, the presently disclosed system may perform ortho mapping and/orstereo mapping of the pictures taken at 1910. Ortho Mapping is a processthat corrects for geometric distortions inherent in remotely sensedimagery to produce ortho imagery products, raw images, and/or Orthoimagery products. Stereo mapping is a stereoscopy-enabled map thatprovides stereo vision through a stereo model, which is composed of twoimages of the same geographic area taken from two locations. Binocularstereo vision facilitates better image interpretation than singular monovision and depth resolution.

At 1930, the presently disclosed system may perform image mensurationanalysis of the pictures taken at 1910. Image mensuration appliesgeometric rules to determine distance, area of a 2-dimensional or3-dimensional surfaces using the information obtained from lines andangles. It may also includes measuring the height and absolute locationof a feature. According to some embodiments, image mensuration analysismay be used to identify landmarks on patient's wound and or burn. Theselandmarks may be used to overlap, overlay, and/or align differentimages.

At 1932, the presently disclosed system may perform perspectivetransformation analysis of the pictures taken at 1910. PerspectiveTransformation (Homography) may be used to change the perspective of agiven image or video for getting better insights into the requiredinformation. In Perspective Transformation, one or more points areselected on the image from which to gather information by changing theperspective. Perspective transformation may also be used to remove imagedistortion.

At 1934, the presently disclosed system may perform image processingand/or segmentation analysis of the pictures taken at 1910 to analyzeimages pixel by pixel for variety of features.

According to some embodiments, presently disclosed system may performfluorescence imaging using wavelength 1903 and wavelength 1905 at any ofthe wavelength described below to test for biological markers, bacteria,infection, tissue, etc. For example, presently disclosed system mayperform fluorescence imaging using violet light where wavelength 1903 isat 405 nm and UVA light where wavelength 1905 is at 365 nm.

Endogenous fluorophores are label-free biological markers of human cellsand tissues as well as bacteria that intrinsically emit fluorescenceupon excitation by a convenient wavelength. This phenomenon is calledautofluorescence (AF). AF or intrinsic fluorescence biomarkers includereduced nicotinamide adenine dinucleotide (NAD(P)H), oxidized flavins(FAD and FMN), porphyrins, collagen, elastin, and other metabolites.These fluorophores have excitation maxima of 325-500 nm and emissionmaxima at longer wavelength (390-700 nm) regions of the spectrum. Themajority of tissue fluorophores, such as collagen crosslinks, NADH,elastin, oxidized flavins, and porphyrins, may be excited by ultravioletA (UVA: 355 nm). Bacterial intrinsic biomolecules, including NADH andFAD, also fluoresce following the excitation by UV light. Collagen,porphyrins, and pyoverdines have excitations at 405 nm (violet light)and emissions at 420-700 nm.

Extracellular matrix proteins of connective tissue (e.g., collagen,elastin, and fibrin) fluoresce in the green and yellow regions of thevisible spectrum after excitation at 405 nm. The color shade is varieddepending on the relative density of collagen, elastin, and their degreeof cross-linking. A higher concentration of melanin in darker skin tonesattenuates the tissue green fluorescence. Slough with high levels offibrin displays bright green after excitation at 405 nm while flaky skinis green with thin white edges indicating the flakes. Tendons and boneswith the highest levels of collagen fluoresce very bright green toglowing white. Hemoglobin absorbs the violet light resulting in the darkblack or maroon color of the blood or highly vascular tissue. Dead ornecrotic tissues are greatly decomposed and black in reflectance andviolet light images.

The amounts of fluorophores are variable based on bacterial types andtheir microenvironments⁴⁹. At critical bacterial loads (≥10⁴ CFU/g),most pathogenic bacteria emit red fluorescence, although Pseudomonasaeruginosa can be indicated by cyan fluorescence upon excitation at 405nm. Siderophore pyoverdines are bacterial iron-chelating molecules thatfacilitate microbial colonization and infection. Pyoverdine is theextracellular siderophore secreted by Pseudomonas aeruginosa and emitsblue-green (cyan) at 430-530 nm with the emission peak at 455 nm uponmaximal excitation at 395 nm. Porphyrins are intermediate molecules inthe bacterial and mammalian heme biosynthetic pathway. Bacterialendogenous porphyrins are associated with the virulence of pathogens andemit red fluorescence when excited by UV and violet lights. Manygram-positive and gram-negative bacteria produce porphyrins, which emitfluorescent at two maxima of 618-620 and 680 nm upon excitation at 405nm. NADH/NAD(P)H, a metabolic coenzyme, has excitation maxima at 350/380nm and emission at 400-500 nm with a strong peak at 470 nm producingblue fluorescence. The excitation and emission wavelengths are variableand depend on the microbial strain and metabolic state. Flavins orflavoproteins (FAD) excite at 420-500 nm and emit green fluorescence at500-600 nm with a strong emission peak around 525 nm. Due to the higherconcentration of NADH and flavins, gram-negative bacteria can bedifferentiated from gram-positive microorganisms. Using multipleexcitation wavelengths, gram-negative bacteria can be detected in greenwhile gram-positives emit red fluorescence.

Referring to FIG. 17 , a method 2000 is shown according to someembodiments presently disclosed. The method 2000 may be used to generateone or more reports pertaining to patient's wound and/or burn based onanalyzes of pictures taken at 1810, and/or taken at 1910, and/or storedin the memory 74 of the device 10, and/or stored in the memory of theserver 95, and/or served in the memory of the server 90. According tosome embodiments presently disclosed, the method 2000 may utilizemachine learning and/or artificial intelligence to analyzes of picturesand generate one or more reports pertaining to patient's wound and/orburn. According to some embodiments presently disclosed, the picturestaken at 1810, and/or taken at 1910, and/or stored in the memory 74 ofthe device 10, and/or stored in the memory of the server 95, and/orserved in the memory of the server 90 are part of machine learningdatabase 2005. According to some embodiments presently disclosed,presently disclosed system utilizes patients' electronic health records(i.e. medical records) 2003 and the database 2005 in the method 2000.

According to some embodiments presently disclosed, presently disclosedsystem utilizes patients' electronic health records (i.e. medicalrecords) 2003, the database 2005, and kernel at 230 to evaluatediagnostic meta data.

The operator turns on and logs into a presently disclosed system runningon the device 10. The operator enters patient's information (i.e.patient's data, and/or patient's record) Patient's information orportion of patient's information may be located in memory 74 of thedevice 10, a memory of the server 95 and/or a memory of the server 90.Patient's information may include patient's medical record(s) and/orpatient's personal information. According to some embodiments, presentlydisclosed system or portion of the presently disclosed system may bestored in memory 74 of the device 10, a memory of the server 95 and/or amemory of the server 90. According to some embodiments, presentlydisclosed system or portion of the presently disclosed system may beprocessed by the processor modules 65 of the device 10, a processor ofthe server 95 and/or a processor of the server 90.

The patient's information may be viewed on the display 70 of the device10 or on another external screen. According to some embodiments,patient's information includes past tests performed by medicalprofessionals, treatments performed and completed by medicalprofessional, treatments that were started and not completed by medicalprofessionals, and/or treatments that were not started by medicalprofessionals. The operator may make changes to patient's information,updated patient's information, add new information (i.e. data) to thepatient's record.

At 2002, the presently disclosed system may confirm that records 2003and database 2005 are available. At 2006, the presently disclosed systemperforms computation and analysis of the records 2003 and database 2005.The presently disclosed system may perform computations using, fully orin part, the processor module 65 of the device 10, and/or a processor ofthe server 95, and/or a processor of the server 90.

At 2016, the presently disclosed system may perform wound classificationanalysis based on the one or more pictures in the database 2005. Medicalclinicians classify wounds based on whether or not patient's skin isbroken, whether or not bone and/or tendons are exposed. FIG. 18 depictsdifferent classification grades for ulcer and different classificationgrades for ischemia. Based on the analysis of the database 2005, andresults of methods 1800, and/or 1900, presently disclosed system assignsone or more classification grades to patient's wound and/or burnAccording to some embodiments, presently disclosed system generates areport identifying classification grades for patient's wound and/orburn, and may suggest treatment to prevent the wound from getting worse.

At 2018, the presently disclosed system may perform ischemia analysisbased on the one or more pictures in the database 2005 to determine ifenough blood and/or oxygen is being supplied to patient's wound and/orburn Dark skin areas around patient's sound and/or burn signify lack ofoxygen being supplied to the patient's wound and/or burn. According tosome embodiments, presently disclosed system generates a reportidentifying ischemia, and may suggest treatment to prevent the woundfrom getting worse.

At 2020, the presently disclosed system may overlay images taken at 1822over images taken at 1922 to generate a picture of the patient's woundand/or burn and show if there is any infection. Overtime, picturesgenerated at 2020 may be compared by the presently disclosed system toeach to determine of the size of the wound is getting smaller or bigger,and/or if the amount if infection is increasing or decreasing.

At 2022, the presently disclosed system may perform homology/morphologyanalysis based on the one or more pictures in the database 2005 todetermine shape and/or depth of the patient's wound and/or burn.

At 2024, the presently disclosed system may provide treatment options(i.e. report) based on the result at 2016, 2018, 2020, and/or 2022. At2026, the presently disclosed system may provide plots statistics ofhealing progress of the patient's wound and/or burn.

At 2028, the presently disclosed system may use segmentation maskinganalysis to analyze pictures in database 205 pixel by pixel. At 2030,the presently disclosed system may use, for example Convolutional neuralnetwork (CNN) kernel engine to analyze images analyzed at 2028. At 2032,the presently disclosed system may use confusion matrix analysis toevaluate and/or determine number of false positives and/or falsenegatives results. At 2034, the presently disclosed system may optimizefor faster processing.

According to some embodiments, the presently disclosed system performssegmentation masking analysis at 2028, apply kernel engine at 2030,apply confusion matrix at 2032, and/or optimization at 2034 on thedatabase 2005 to generate wound classification at 2016, ischemia reportat 2018, overlay at 2022, provide treatment options at 2024, and/orprovide plot statistics at 2026.

Referring to FIG. 19 , a method 2500 is shown according to someembodiments presently disclosed. According to some embodiments, themethod 2500 may be used to assign billing code(s) to patient's woundand/or burn based on the wound condition identified, for example, at2016. Billing codes may be insurance specific billing codes or Medicareand Medicaid Services (CMS) billing codes.

According to some embodiments, presently disclosed system and method2500 collect data from patient records at 2003, wound classification at2016, and/or treatment matrix at 2024 and correlates this data tobilling code library 2506. The billing code library 2506 may beperiodically updated to have the most updated list of codes. The billingcode library 2506 may be stored in memory 74 of the device 10, in memoryof the server 90, and/or memory of the server 95. According to someembodiments, method 2500 has access to treatment matrix 2507 generated,for example, at 2016, 2018, 2022, 2024 and/or 2026.

At 2510, the presently disclosed system obtains progress and treatmentdata from records 2003, database 2005, matrix 2507. At 2512, thepresently disclosed system generates a report containing healingprogress of the patient's wound and/or burn, treatment for the patient'swound and/or burn, and a billing code that from library 2506 thatcorresponds to treatment of the patient's wound and/or burn and/orbilling code for examination and/or testing of the patient's woundand/or burn. The report at 2512 may be generated based on results frommethod 1800, 1900, and/or 2000. The billing code(s) are added to thereport at 2512 from the library 2506 at 2520.

At 2514, the presently disclosed system may provide metrics and staticsgenerated based on results from method 1800, 1900, and/or 2000. At 2514,the presently disclosed system may provide notes made by operator of thedevice 10 and/or any other medical professional. At 2518, the presentlydisclosed system may transmit patient's record to another medicalprofessional or insurance company.

At 2514, the presently disclosed system may determine if the correctbilling code was properly applied. At 2514, the presently disclosedsystem may determine if patient should be referred to another medicalprofessional for additional treatment(s). At 2514, the presentlydisclosed system may allow the one or more generated reports to beemailed, printed, or otherwise transmitted. At 2514, the presentlydisclosed system may provide one or more reports to 3^(rd) party systemfor billing, hospital or another medical professional.

According to some embodiments, presently disclosed system uses pulsatinglight to remove ambient light fluorescence (infection, metabolism, andfluorescence endogenous or exogenous). The image can be easilycontaminated by the presence of ambient light while capturingfluorescence images. Ambient light refers to the light from asurrounding scene or a situation. Light that may come in through awindow, from a ceiling lamp, or other source of illumination are allexamples of ambient light. To obtain an uncontaminated fluorescenceimage it may be necessary to remove the ambient light. The fluorescencesignal captured in images or by spectroscopy measurements has diagnosticor analytical significance when the light that is not from thebiological tissue is removed.

There are multiple methods to remove undesired ambient light from animage or a spectroscopy measurement of a biological tissue beinganalyzed. One exemplary way is to capture an image of the subject withthe fluorescence excitation light turned off and then capture an imagewith the fluorescence excitation light turned on. The background ambientlight can be removed from the fluorescence image by subtracting theimage with no excitation illumination from the image with excitationillumination. Ambient light can also be subtracted by applyingweighted-subtraction and machine-learning approaches.

According to some embodiments, presently disclosed system may change theimage capture system exposure time to accurately capture both theambient light and fluorescence images without them being too dark or toolight. These images can be scaled for the correct or relevant amount ofsubtraction according to their exposure times. A control or calibrationtarget and other apriori information obtained from the ambient lightusing an image of a reference surface can be used to appropriately scalethe exposure time or speed the image acquisition time.

According to some embodiments, presently disclosed system provides multibiomarker measurements in a handheld device 10. According to someembodiments, presently disclosed system is a multi-mode portable imagingsystem comprising the optical measurement modes of tissue fluorescence,tissue reflectance, thermal imaging, and three-dimensional imaging.Tissue fluorescence is a result of the absorption by tissue molecules ofa higher energy photon and the emission of a lower energy photon. Therelative response of different wavelengths illuminating and emitted bytissue fluorophores can allow identification or classification of thetissue. Tissue reflectance refers to the portion of illumination lightremitted from a tissue. Light remitted from a tissue can include lightremitted by diffuse or specular reflection from the surface of thetissue or light remitted through scattering processes (e.g., reflectancethrough backscatter). Thermal imaging is the measurement of infraredlight emitted from tissue according to its temperature.Three-dimensional imaging refers to methods to capture an image of anobject that measures the topology or shape of the object. Examples ofthree-dimensional image capture devices include laser scanning systems,structured illumination systems, photogrammetry, and LIDAR 50 imagecapture systems. To detect multiple biomarkers relevant to aphysiological area of interest [e.g., a limb or an area of skin], bothlight sources 55 and optical sensors 25, 30 can be used for fluorescenceor reflectance imaging.

Light sources 55 or optical sensors can be applied for both reflectanceand fluorescence imaging. Selection of light source 55 wavelengths orselection of optical sensors optical filters may be applied in variouscombinations to capture fluorescence or reflectance information ofinterest.

According to some embodiments, presently disclosed system comprises thehandheld device 10 that is capable of measuring optical characteristicsof wounds for diagnostic and treatment purposes. According to someembodiments, the handheld device can identify the location as well asthe concentration of multiple metabolic biomarkers, such as NADH andFAD. FAD and NADH are autofluorescent coenzymes of the mitochondrialrespiratory chain whose levels increase with high blood glucose.Additionally, the device 10 may identify the location andcharacteristics of tissue structural components, such as collagen. Thedevice 10 may also detect the presence of tissue infection, such asporphyrin-producing bacteria found in biofilms and measure bacterialload of a wound. According to some embodiments, presently disclosedsystem measures the oxygenation status of tissue (the relative amount ofoxy- and deoxyhemoglobin) and elucidate other structural features suchas blood vessel networks.

According to some embodiments, presently disclosed system provides 3Dmapping of wound and body anatomy, i.e. calluses, moles, scars, skinconditions etc. According to some embodiments, presently disclosedsystem comprises multimode imaging system that reconstructs thethree-dimensional geometry and topology of a limb or surface feature,such as a callus, mole, scar, skin chronic conditions (psoriasis,eczema, etc), wound, ulcers, limb, burn, military related wound,chemical burn etc.

According to some embodiments, multimode imaging or multimode pointspectroscopy measurements of skin can be used for many kinds of skinillnesses or injuries, including skin chronic conditions (e.g.,psoriasis, and eczema, atopic dermatosis). Multimode imaging can detectdistinct structural features related to these conditions, such asdisruption and alteration of skin surface texture, and raised or unevenskin areas. According to some embodiments, presently disclosed systemmay be used to identify such abnormalities and other characteristics,such as color, oxygenation status, metabolic biomarkers, and structuralfeatures derived from fluorescence and reflectance imaging as well as 3Dimaging modes.

According to some embodiments, the multimode imaging system can beapplied for imaging of both chronic and acute wound types. Diabetic andpressure ulcers are examples of chronic wounds, while acute woundsinclude accidental wounds, such as weapon assaults, automobile accidentsor workplace injuries. Acute wounds also include surgical wounds (e.g.,intentional skin incisions or excisions for treatment purposes).According to some embodiments, presently disclosed system can alsodocument the characteristics, shape, and structure of the wound. This isparticularly useful for billing for medical procedures. The gradualprocess of wound healing can also be documented by imaging overtimechanges in wound parameters, including dimensions, quantification ofcollagen regeneration and cross-linking, metabolic activity of theperi-wound area, and other characteristics of skin regeneration.

According to some embodiments, presently disclosed system provides imageregistration of multiple biomarkers on tissue. Presently disclosedmultimode imaging systems may use multiple cameras (i.e. opticalsensors) or other image capture systems, such as 3D imaging.

According to some embodiments, presently disclosed system correlatesinformation captured in each of the multiple modes with its real-worldlocation on the human anatomy. According to some embodiments, presentlydisclosed system registers and maps the data to the correct location onthe resulting clinical model of the surface of the skin or anatomicalfeature being examined. According to some embodiments, presentlydisclosed system translates and/or scales images so that they overlapexactly. Scaling of images can compensate for camera characteristicsthat create different image sizes or different fields of view to mapthem to a common spatial scale. Translation of images involves shiftingand/or rotating the x and y coordinates of an image so that image pixellocations are mapped to their common features in another image.According to some embodiments, presently disclosed system registersmultiple two-dimensional images by translation and warping to accuratelyregister them to common physiological features on a 3D reconstructionmodel of the tissue.

According to some embodiments, presently disclosed system determinesoxygenation independent of skin color. One of the characteristic ofmultimode imaging is the ability to use the different modes of opticalmeasurement to augment data from the other modes. For example, melanininterferes with the accuracy of oxygenation measurements when image iscaptured through the skin, particularly around wounds. Historically,many optically-based tissue oxygenation measurements have not workedwell on people with darker skin. This is because the melanin in thebasement membrane and in the epidermis absorbs light broadly. Areashighly pigmented with melanin produce confusion in interpreting theoptical measurement of oxyhemoglobin.

According to some embodiments, presently disclosed system compensatesfor the effect of melanin absorption on the determination of theoxy/deoxy hemoglobin ratio. By using multimode measurements, thepresently disclosed system can determine the relative contributions ofdifferent chromophores in the tissue. Multimode measurements may also beused to compensate for variability of the quality of an individual modeof measurement through cross-referencing. For example, understanding themelanin concentration through multimode imaging helps to subtract it ortake it into account in some other way in order to produce a moreaccurate measurement of underlying fluorescence. Fluorescencemeasurements at multiple wavelengths can be scaled in response to therelative melanin contribution. To investigate blood vessel networks,reflectance imaging (sometimes accompanied with polarization) can beused. Blood vessel networks can also be detected using fluorescenceimaging. The intensity to which fluorescence is quenched by the bloodvessel networks can be used to augment reflectance and backscatteredlight imaging. In this way, the multiple modes may determine thecontribution of the many different biomarkers in tissue and to map themappropriately on the surface of the skin.

Another mode of imaging in the multimode system provides 3D imaging ofthe skin. This mode determines the impact of the surface topology of theskin, such as the curvature of the skin, on the measurements being made.For example, diffuse reflectance and backscatter from skin is verydependent on the angle of incidence between the light hitting the skinand the light coming back. This can affect the relative intensity. Bycompensating for the three-dimensional shape and understanding the angleof the skin between the imaging device and the tissue, presentlydisclosed system may correct for the relative response using, forexample, Lambert's law and the cosine response.

According to some embodiments, presently disclosed system provides imageanalysis of multi-biomarkers. According to some embodiments, presentlydisclosed system offers additional advantages in that information gainedfrom the image registration on a three-dimensional surface model can befurther applied using machine learning techniques. According to someembodiments, presently disclosed system provides artificial intelligencemachine learning algorithms for detection, localization, andclassification of important skin features. According to someembodiments, presently disclosed methods can be deployed in real time onimaging devices to recognize and identify physiological conditions.

According to some embodiments, presently disclosed system provides 3dmeasurement (stereoscopic imaging, LIDAR, etc), Range finder, gyroscope,motion sensor, wound objective measurements, longitudinal wound/lesionsize analysis, and/or topographic analysis. One of the modes of imagingused in the presently disclosed system may be the 3D measurement. Thiscan be accomplished in multiple ways for application in multimodeimaging. For instance, while multiple images of a surface are taken, aLIDAR system could measure the distance from the camera to the surfacebeing imaged. Other sensors in the presently disclosed system couldmeasure the angle, orientation of the camera, and any movements sinceprevious measurements, to understand the circumstances in which theseimages are captured in 3D space. This information can further be used toreconstruct the 3D image of the biological tissue being examined. Inanother mode of imaging, there can be multiple cameras (i.e. opticalsensors) in the presently disclosed system. These cameras can besensitive to different biomarkers and different optical characteristicsof the tissue. These cameras can capture a spatial image of the tissue,simultaneously. Since these cameras are offset from one another and maybe side-by-side, stereoscopic imaging can be further used to reconstructthree-dimensional structure of the features in the image. Other modes ofstereoscopic imaging can also be accomplished using cameras with sensorsthat incorporate multiple pixels that are directionally sensitive. Thepixels are offset from one another and can generate two image fields ofa system from slightly different angles that can be applied tostereoscopic imaging and photogrammetry.

Another method of three-dimensional imaging is scanning LIDAR imaging.This method comprises a raster scanning laser and works based on theangle and time when the sensor receives the laser signal and that isused to measure the distance between the object and the sensor. Anothermethod of three-dimensional imaging uses structured light (e.g., DLPscanning system) projected to create arrays of lines or points on theobject being imaged, can measure the surface topology of the objectbeing imaged from the distortion of the reference array, while images ofthe object surface are captured using IR and/or other wavelengths.

According to some embodiments, presently disclosed system may usemultiple sensors and multiple multimode images for reconstruction into amultidimensional image cube or hyper cube incorporating functional,spatial and structural, and even temporal dimensions. This provides asingle data analysis framework that can be used to interpret theprogress of a wound or lesion or burn on an anatomical feature such as afoot by examining dimensional and structural changes, metabolic changes,blood vessel network changes, and healing processes such as collagenremodeling.

According to some embodiments, longitudinal measurements may be used toexamine the progress of wound or lesion. The measurements can be used inproactive diagnostic way to capture wound that is progressing to becomemore severe. Also, they can be applied for monitoring the wound healing,as it becomes less severe and heals.

According to some embodiments, presently disclosed system may useBlockchain, trust certificate of image analysis data of multiplebiomarkers. One aspect of the development of electronic health recordsand the review of images across institutions and over time, isunderstanding and being certain that the image information is accuratelyassociated with the time in which it was captured, and the patient fromwhich it was captured. Blockchain provides a method to create trustcertificates of image analysis data that can improve data security andthe correct interpretation and understanding of the progression of awound or the healing of a wound following surgery. This makes sure thata clinician reviewing any image data or patient information has anaccurate representation of that data. It can also be useful in assigningcosts and times for reimbursement and an assessment of appropriatetreatment plans.

According to some embodiments, presently disclosed system may useFederated learning, Machine learning (ML), Artificial intelligence (AI)confidentiality of image analysis data of multiple biomarkers. Accordingto some embodiments, presently disclosed system includes the handheldmultimode imaging device and a system component for managing, storing,and aggregating data incorporated in the system. According to someembodiments, presently disclosed system comprises a system processorwith data storage that can be wirelessly connected to the handheldmultimode imaging device 10. This system processor can be a virtualsystem, such as a cloud server, or a physical system, such as a laptopcomputer. The system processor provides operator interfaces and methodsto review and input data. The system also provides the ability tocommunicate with multiple handheld imaging devices. The system furtherprovides the ability to communicate between multiple servers eachhosting multiple handheld imaging devices.

According to some embodiments, presently disclosed system provides theability to collect and analyze data to improve the accuracy of theimaging algorithm over time. According to some embodiments, presentlydisclosed learning systems can be hosted solely in the system processor,or they can be distributed among multiple system processors of multiplecustomers. According to some embodiments, presently disclosed systemprovides the ability to distribute learning systems across multiplecustomer system processors while maintaining the privacy of their datausing, for example, the method of “Federated Learning” to improve theaccuracy of the imaging algorithm over time. Federated learning is amethod of aggregating data to improve machine learning algorithms,without transferring the data to an external machine learning server.According to some embodiments, presently disclosed system provides thesoftware that will reside on the data repository of the person using themultimode imaging system. It will analyze images and perform machinelearning on those images to improve the parameters of a deployable imageanalysis algorithm, without transferring any confidential data to wherethe algorithm will be deployed.

According to some embodiments, presently disclosed system providestelehealth integration and/or integration into electronic health recordof image analysis data of multiple biomarkers. Example of biomarkers aretemperature, oxygenation, vascularization, redox ratio, infection maps.

According to some embodiments, presently disclosed system providessource of light coherent and incoherent, scanning mechanical andnon-mechanical.

According to some embodiments, presently disclosed system providesintegration of wound imaging, and swab based bacterial analysis (McGillor similar lab on chip methods).

According to some embodiments, presently disclosed system providesintegration of “field of view” imaging with point measurement methods.Field of view imaging is the measurement of an area of an anatomicalregion. While field of view imaging can be challenging and costly tomake measurements with high specificity, it can often be optimized forgood sensitivity over a large field of view. In contrast, pointmeasurement systems can provide lower cost and more accurate andin-depth information about a single point on an anatomical surface. Theintegration of field of view imaging with point measurement, uses thespeed and sensitivity of the multimode imaging system to locate an areaof interest where a point measurement could be taken. This may improvemany aspects of care. Examples of point measurements include swab-basedmeasurements, where a swab is applied to a surface followed bymicrobiological culture, DNA analysis, or other chemical forms ofanalysis. Point measurements can also include point spectroscopy, wherean optical measurement is taken and analyzed for in-depthcharacteristics using spectral properties. Point spectroscopy can alsocomprise a multimode point measurement spectroscopy system. Another formof point measurement is tissue biopsy, in which a small sample of tissueis extracted and analyzed for histological features such as cellstructure. Another form of point measurement is liquid sampling, wheresmall samples of extracellular fluid, blood or lymph are extracted forfurther kinds of analysis. Other types of point measures can includemonitoring of temperature and skin conductivity as well as ultrasoundmeasurements.

According to some embodiments, presently disclosed system providesintegration of multimode spectroscopy and multimode imaging for morespecific analysis.

According to some embodiments, presently disclosed system can be usedboth for sensing and for other purposes such as treatment. Methods ofusing light for treatment can include activating phototherapeutic drugs,providing specialized combinations of illumination wavelengths thatmaybe needed to trigger polymerization or other forms of activation ofpolymers or polymer substrates (e.g., curing UV epoxies), or opticaltriggering of drug release by initiating breakdown of drug encapsulationsystems.

The same illumination light used to capture tissue information can alsobe applied in certain cases to activate therapeutic agents, such asphototherapy drugs, in the multimode optical imaging systems.

According to some embodiments, presently disclosed system providesfinger print scanners, barcode reader, facial recognition for operatorand/or patient part of tamper proof mechanism. According to someembodiments, presently disclosed system can reliably identify thepatient, the system's operator, or attending physician when images arecaptured. Ways to accomplish this include incorporation of biometricrecognition systems, such as fingerprint scanners and/or facialrecognition for operators, patients, and attending clinicians. It canalso include barcode reading systems or RFID reading systems. Thisinformation can be incorporated with the image data for futureverification of attending personnel and so that the measurements areaccurately linked to the correct patient. According to some embodiments,presently disclosed system may provide temper proofing to assure thatinappropriate personnel will not be able to use the technology.

According to some embodiments, presently disclosed system providesclassification algorithm to authenticate reimbursement/billingeligibility and/or approval procedure. When assessing skin injuries,presently disclosed system may classify the nature of an injury or achronic problem in order to authenticate reimbursement, billingeligibility, and approval of procedures. For instance, differentiatingbetween treatment for burns versus treatment for chronic skinconditions, such as psoriasis or eczema. According to some embodiments,presently disclosed system allows to accurately document and classifythe nature of the skin injury being treated provides a disincentive forthis type of falsification.

According to some embodiments, presently disclosed system may be usedfor ulcer prevention, detection of pre-inflammation, etc. According tosome embodiments, presently disclosed system may be used for limb 3Dtopography, skin health cutaneous and subcutaneous properties, fittingprostatic. According to some embodiments, presently disclosed system maybe used for plastic surgery, wound hip replacement, spine surgerywounds. According to some embodiments, presently disclosed system may beused for skin tissue, wound, and or scar analysis of impact of externalobjects or exposure with skin condition. According to some embodiments,presently disclosed system may be used for postmortem skin tissueanalysis.

As Point Feature representations/Histogram (PFH) go, surface (i.e.surface containing wound and/or burn) normal and curvature estimates aresomewhat basic in their representations of the geometry around aspecific point. The goal of the PFH formulation is to encode a point'sk-neighborhood geometrical properties by generalizing the mean curvaturearound the point using a multi-dimensional histogram of values. Thishighly dimensional hyperspace provides an informative signature for thefeature representation, is invariant to the 6 degrees of freedom pose ofthe underlying surface, and copes very well with different samplingdensities or noise levels present in the neighborhood. The followingfeatures of FPH may be used in 3D mapping of the wound features: 1) FastPoint Feature Histograms (FPFH) for 3D registration, 2) optimizations ofthe PFH computations that reduce runtime drastically by reordering thedataset and caching previously computed values; and/or 3) The PFHselection criterion at a given scale is motivated by the fact that in agiven metric space, one can compute the distances from the mean PFH of adataset to all the features of that dataset. More details can be foundin an article by R. B. Rusu, N Blodow and M. Beetz, “Fast Point FeatureHistograms (FPFH) for 3D registration,” 2009 IEEE InternationalConference on Robotics and Automation, 2009, pp. 3212-3217, doi10.1109/ROBOT.2009.5152473, which is incorporated herein by reference inits entirety.

A digital orthophoto Imagimg (DOI)—or any orthoimage—is acomputer-generated image of wound imaging photograph in whichdisplacements (distortions) caused by terrain relief and camera tiltshave been removed. It combines the image characteristics of a photographwith the geometric qualities of a map. Ortho Imaging techniques,borrowed from GIS and other fields may be utilized for efficient woundmapping. DOI may be used: 1) to measurement of 3D coordinates usingorthorectified stereo images; 2) for multispectral analysis,visualisation and animation, and improved manual feature classification;3) for detection and correction of DTM (Digital Terrain Model) errors;and/or 4) for processing input images, writing the ortho-image channels,and grey level interpolation More details can be found in an article byEmmanuel P. Baltsavias, Digital ortho-images—A powerful tool for theextraction of spatial- and geo-information, ISPRS Journal ofPhotogrammetry and Remote Sensing, Volume 51, Issue 2, 1996, Pages63-77, ISSN 0924-2716, which is incorporated herein by reference in itsentirety.

Image-based data integration in eHealth and life sciences is typicallyconcerned with the method used for anatomical space mapping, needed toretrieve, compare and analyze large volumes of biomedical data. Inmapping one image onto another image, a mechanism is used to match andfind the corresponding spatial regions which have the same meaningbetween the source and the matching image. Image-based data integrationis useful for integrating data of various information structures. Thereis a broad range of issues related to data integration of variousinformation structures, review exemplary work on image representationand mapping, and discuss the challenges that these techniques may bring.Spatial relations as mapping primitives. Metric relations describe thevalue of the quantitative distance between two spatial entities.Distance can be measured, and it specifies how far is the entity awayfrom the reference entity. Directional relations are usually describedbetween two spatial entities that do not overlap. Approximation forthese relations can be done by comparing entities representative points(also called centroid) or their minimum bounding boxes. These relationsare often described based on cardinal directions between two spatialentities. Fiducial points: image processing-based mappings. Two types ofmapping primitives using spatial relations and fiducial points.Ontology-based mappings may use spatial relations, whilst imageprocessing-based mappings may use fiducial points. These two types ofmapping primitives are able to determine corresponding anatomicalregions across images. More details can be found in an article by MohdZaizi N J, Awang Iskandar D N. Using image mapping towards biomedicaland biological data sharing. Gigascience. 2013 Sep. 23; 2(1):12 doi:10.11862047-217X-2-12. PMID: 24059352, PMCID: PMC3852063, which isincorporated herein by reference in its entirety.

Measuring ground features in imagery, called “image mensuration”, is afunction in many image interpretation or feature compilation andapplications. Image mensuration is defined as applying geometric rulesto determine distance, area of a 2-dimensional or 3-dimensional surfacesusing the information obtained from lines and angles. It also includesmeasuring the height and absolute location of a feature Mensuration(aka: repeat, period, Retake etc.) mapping/analysis include pointdistance, area, centroid, height, shadow height, 3D point, 3D area andvolume. These will increase the accuracy of measurements in 3Dmorphological measurements and mapping. According to some embodiments,these techniques are being applied to wound feature analysis andmapping. More details can be found in an article by Fraser, Clive S. etal. “Precision Target Mensuration in Vision Metrology.”, O. Univ. Prof.Dipl.-Ing. Dr. techn. Karl Kraus E122, Institut für Photogrammetrie undFernerkundung, TU-Wien, which is incorporated herein by reference in itsentirety.

When human eyes see near things they look bigger as compare to those whoare far away. This is called perspective in a general way. Whereastransformation is the transfer of an object etc from one state toanother So overall, the perspective transformation deals with theconversion of 3d world into 2d image. Some of the data augmentationtechniques used for images are: Position augmentation Scaling. Cropping.Flipping. Padding. Rotation. Translation. Affine transformation. Coloraugmentation. Brightness Contrast. Saturation. These applications aresuitable for AI/ML and automated image processing presently disclosed.More details can be found in an article by K. Wang, B. Fang, J. Qian, S.Yang, X. Zhou and J. Zhou, “Perspective Transformation Data Augmentationfor Object Detection,” in IEEE Access, vol 8, pp. 4935-4943, 2020, doi:10.1109/ACCESS.2019.2962572, which is incorporated herein by referencein its entirety.

Convolution is a mathematical operation which is fundamental to manycommon image processing operators. Convolution provides a way of‘multiplying together’ two arrays of numbers, generally of differentsizes, but of the same dimensionality and used for image reconstructionin various imaging modality. The images reconstructed byconvolution/filtering is superior in quality compared with thosereconstructed by existing. More details can be found in an article byMondal P P, Rajan K, Ahmad I. Filter for biomedical imaging and imageprocessing. J Opt Soc Am A Opt Image Sci Vis. 2006 July; 23(7):1678-86.doi 10.1364/josaa 23.001678 PMID: 16783431, which is incorporated hereinby reference in its entirety.

More details about multimode imaging for skin analysis and reducingeffect of melanin on measurement of tissue oxygenation can be found inan article by F Vasefi, et al, Polarization-sensitive hyperspectralimaging in vivo: a multimode dermoscope for skin analysis, Scientificreports 4 (1), 1-10, (2014), which is incorporated herein by referencein its entirety. May also be found in an article by F Vasefi, et al,Separating melanin from hemodynamics in nevi using multimodehyperspectral dermoscopy and spatial frequency domain spectroscopy,Journal of biomedical optics 21 (11), 114001 (2016), which isincorporated herein by reference in its entirety. May also be found inan article by F Vasefi et al., Toward in vivo diagnosis of skin cancerusing multimode imaging dermoscopy: (II) molecular mapping of highlypigmented lesions, Imaging, Manipulation, and Analysis of Biomolecules,Cells, and Tissues XII, (2014), which is incorporated herein byreference in its entirety.

More details about various hyperspectral and multimode spectral imagingfor skin analysis can be found in an article by F Vasefi et al.,Hyperspectral and multispectral imaging in dermatology, Imaging inDermatology, 187-201, (2016), which is incorporated herein by referencein its entirety.

More details about combination of mobile imaging with clinical scannerfor skin and wound monitoring can be found in an article by N MacKinnonet al, Melanoma detection using smartphone and multimode hyperspectralimaging, Imaging, Manipulation, and Analysis of Biomolecules, Cells, andTissues IX (2016), which is incorporated herein by reference in itsentirety. May also be found in an article by N Alamdari et al, Effect oflesion segmentation in melanoma diagnosis for a mobile healthapplication, Frontiers in Biomedical Devices 40672, V001T12A005, (2017),which is incorporated herein by reference in its entirety.

More details about using time-resolved fluorescence imaging anddetection of multiple fluorescence biomarkers can be found in an articleby F. Vasefi et al, Real time optical biopsy: time-resolved fluorescencespectroscopy instrumentation and validation, Journal of biomedicaloptics 21 (11), 114001 (2016), which is incorporated herein by referencein its entirety.

More details about identifying foot and body landmark and imageregistration can be found in an article by F Akhbardeh et al, Towarddevelopment of mobile application for hand arthritis screening, 201537th Annual International Conference of the IEEE Engineering in Biology(2015), which is incorporated herein by reference in its entirety Mayalso be found in an article by M Amini et al, Validation of hand andfoot anatomical feature measurements from smartphone images, Imaging,Manipulation, and Analysis of Biomolecules, Cells, and Tissues XVI,(2018), which is incorporated herein by reference in its entirety.

More details describing multimode imaging being complementary measuringbiomarkers with more accuracy can be found in an article by F Vasefi etal, Multimode hyperspectral imaging for food quality and safety,Hyperspectral Imaging in Agriculture, Food and Environment (2018), whichis incorporated herein by reference in its entirety.

More details describing optical imaging scanner calibration using tissuemimicking phantom can be found in an article by F Vasefi et al,Quantifying the optical properties of turbid media using polarizationsensitive hyperspectral imaging (SkinSpect): Two-layer optical phantomstudies, Imaging, Manipulation, and Analysis of Biomolecules, Cells, andTissues XIII (2015), which is incorporated herein by reference in itsentirety.

More details describing computer algorithm detecting lesion boundaryincluding infection and inflammation using smartphone can be found in anarticle by F Vasefi et al, Vanishing point: a smartphone applicationthat classifies acne lesions and estimates prognosis, Imaging,Manipulation, and Analysis of Biomolecules, Cells, and Tissues IX(2016), which is incorporated herein by reference in its entirety. Mayalso be found in an article by F Vasefi et al, A smartphone applicationfor psoriasis segmentation and classification, Imaging, Manipulation,and Analysis of Biomolecules, Cells, and Tissues XV (2017), which isincorporated herein by reference in its entirety.

More details describing computer algorithm automatic detectingfluorescence from bacterial load using machine learning algorithms canbe found in an article by HT Gorji et al, Combining deep learning andfluorescence imaging to automatically identify fecal contamination onmeat carcasses, Scientific Reports 12 (1), 1-11, (2022), which isincorporated herein by reference in its entirety. May also be found inan article by M Sueker et al, Handheld multispectral fluorescenceimaging system to detect and disinfect surface contamination, Sensors 21(21), 7222 (2021), which is incorporated herein by reference in itsentirety. May also be found in an article by HT Gorji et al, Deeplearning and multiwavelength fluorescence imaging for cleanlinessassessment and disinfection in Food Services, Frontier in Sensors(2022), which is incorporated herein by reference in its entirety.

More details describing computer algorithm identifying optimumwavelength for image classification can be found in an article by JChauvin et al, Simulated Annealing-Based Wavelength Selection for RobustTissue Oxygenation Estimation Powered by the Extended ModifiedLambert-Beer Law, Applied Sciences Applied Sciences 12 (17), 8490(2022), which is incorporated herein by reference in its entirety. Mayalso be found in an article by J Chauvin et al, SimulatedAnnealing-Based Hyperspectral Data Optimization for Fish SpeciesClassification: Can the Number of Measured Wavelengths Be Reduced?Applied Sciences 11 (22), 10628 (2021), which is incorporated herein byreference in its entirety. May also be found in an article by J Chauvinet al, Hyperspectral band selection for food fraud application usingself-organizing maps (SOM), SPIE Future Sensing Technologies 11525,201-211 (2020), which is incorporated herein by reference in itsentirety.

More details describing fusion AI computer algorithm combining multimodespectral data analysis for classification can be found in an article byR Duran et al, Multimode hyperspectral data fusion for fish speciesidentification using supervised and reinforcement learning, Sensing forAgriculture and Food Quality and Safety XII 11421, 95-103 (2020), whichis incorporated herein by reference in its entirety.

More details describing calibration and variability analysis offluorescence signal can be found in an article by N Marin et al,Calibration standards for multicenter clinical trials of fluorescencespectroscopy for in vivo diagnosis, Journal of Biomedical Optics 11 (1),014010, (2006), which is incorporated herein by reference in itsentirety. May also be found in an article by BM Pikkula et al,Instrumentation as a source of variability in the application offluorescence spectroscopic devices for detecting cervical neoplasia,Journal of Biomedical Optics 12 (3), 034014 (2007), which isincorporated herein by reference in its entirety. May also be found inan article by J S Lee et al, Design and preliminary analysis of a studyto assess intra-device and inter-device variability of fluorescencespectroscopy instruments for detecting cervical neoplasia, Gynecologiconcology 99 (3), S98-S111 (2005), which is incorporated herein byreference in its entirety. May also be found in an article by BM Pikkulaet al, Multicenter clinical trials of in-vivo fluorescence: are themeasurements equivalent?, Advanced Biomedical and Clinical DiagnosticSystems V 6430, 329-339 (2007), which is incorporated herein byreference in its entirety.

In one or more exemplary designs, the functions described may beimplemented in hardware, software, firmware, or any combination thereof.If implemented in software, the functions may be stored on ortransmitted over as one or more instructions or code on acomputer-readable medium. Computer-readable media includes both computerstorage media and communication media including any medium thatfacilitates transfer of a computer program from one place to another,i.e. may include transitory and/or non-transitory computer readablemedia. A storage media may be any available media that can be accessedby a computer. By way of example, and not limitation, suchcomputer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or otheroptical disk storage, magnetic disk storage or other magnetic storagedevices, or any other medium that can be used to carry or store desiredprogram code in the form of instructions or data structures and that canbe accessed by a computer. Also, any connection is properly termed acomputer-readable medium. For example, if the software is transmittedfrom a website, server, or other remote source using a coaxial cable,fiber optic cable, twisted pair, digital subscriber line (DSL), orwireless technologies such as infrared, radio, and microwave, then thecoaxial cable, fiber optic cable, twisted pair, DSL, or wirelesstechnologies such as infrared, radio, and microwave are included in thedefinition of medium. Disk and disc, as used herein, includes compactdisc (CD), laser disc, optical disc, digital versatile disc (DVD),floppy disk and blu-ray disc where disks usually reproduce datamagnetically, while discs reproduce data optically with lasers.Combinations of the above should also be included within the scope ofcomputer-readable media.

While several illustrative embodiments of the invention have been shownand described, numerous variations and alternative embodiments willoccur to those skilled in the art. Such variations and alternativeembodiments are contemplated, and can be made without departing from thescope of the invention as defined in the appended claims.

As used in this specification and the appended claims, the singularforms “a,” “an,” and “the” include plural referents unless the contentclearly dictates otherwise. The term “plurality” includes two or morereferents unless the content clearly dictates otherwise. Unless definedotherwise, all technical and scientific terms used herein have the samemeaning as commonly understood by one of ordinary skill in the art towhich the disclosure pertains.

The foregoing detailed description of exemplary and preferredembodiments is presented for purposes of illustration and disclosure inaccordance with the requirements of the law. It is not intended to beexhaustive nor to limit the invention to the precise form(s) described,but only to enable others skilled in the art to understand how theinvention may be suited for a particular use or implementation. Thepossibility of modifications and variations will be apparent topractitioners skilled in the art. No limitation is intended by thedescription of exemplary embodiments which may have included tolerances,feature dimensions, specific operating conditions, engineeringspecifications, or the like, and which may vary between implementationsor with changes to the state of the art, and no limitation should beimplied therefrom. Applicant has made this disclosure with respect tothe current state of the art, but also contemplates advancements andthat adaptations in the future may take into consideration of thoseadvancements, namely in accordance with the then current state of theart. It is intended that the scope of the invention be defined by theClaims as written and equivalents as applicable. Reference to a claimelement in the singular is not intended to mean “one and only one”unless explicitly so stated. Moreover, no element, component, nor methodor process step in this disclosure is intended to be dedicated to thepublic regardless of whether the element, component, or step isexplicitly recited in the claims. No claim element herein is to beconstrued under the provisions of 35 U.S.C. Sec. 112, sixth paragraph,unless the element is expressly recited using the phrase “means for . .. ” and no method or process step herein is to be construed under thoseprovisions unless the step, or steps, are expressly recited using thephrase “step(s) for . . . . ”

What is claimed is:
 1. A system for assessing biological tissue, thesystem comprising: an illumination hardware arrangement comprisingtransmission and sensing hardware, the illumination hardware arrangementconfigured to inspect a biological tissue using at least two modes froma group comprising: a three dimensional stereo imaging mode; afluorescence imaging mode; a reflectance imaging mode; and a thermalimaging mode; and processing hardware configured to operate theillumination hardware arrangement according to a protocol comprisinginspection settings of the at least two modes, wherein the processinghardware receives scan results for the at least two modes from theillumination hardware arrangement and identifies attributes of thebiological tissue by constructing a three dimensional dataset from thescan results for the at least two modes and analyzing the threedimensional dataset.
 2. The biological tissue inspection apparatus ofclaim 1, wherein the biological tissue comprises a wound on a patient'sskin.
 3. The biological tissue inspection apparatus of claim 1, whereinthe processing hardware comprises a processor, at least one trainedartificial intelligence module, and at least one classifier.
 4. Thebiological tissue inspection apparatus of claim 1, wherein thebiological tissue comprises a burn on a patient's skin.
 5. Thebiological tissue inspection apparatus of claim 1, wherein the protocolis determined in part based on an identification of particularattributes expected to be associated with the biological tissue whenexamined using the at least two modes.
 6. The biological tissueinspection apparatus of claim 1, wherein the system determines presenceof infection associated with the biological tissue.
 7. The biologicaltissue inspection apparatus of claim 1, wherein the identifiedattributes are infection associated with the biological tissue.
 8. Thebiological tissue inspection apparatus of claim 1, wherein theidentified attributes are metabolic biomarkers associated with thebiological tissue.
 9. The biological tissue inspection apparatus ofclaim 1, wherein the illumination hardware comprises one or morepulsating light sources to reduce ambient light sources for the at leasttwo modes.
 10. The biological tissue inspection apparatus of claim 1,wherein the identified attributes are collagen associated with thebiological tissue.
 11. The biological tissue inspection apparatus ofclaim 1, wherein the identified attributes are oxygenation associatedwith the biological tissue.
 12. The biological tissue inspectionapparatus of claim 1, wherein the system identifies treatment based onthe three dimensional dataset.
 13. The biological tissue inspectionapparatus of claim 1, wherein the system identifies billing code basedon the three dimensional dataset.
 14. The biological tissue inspectionapparatus of claim 1, wherein the system identifies treatment based onthe three dimensional dataset.
 15. The biological tissue inspectionapparatus of claim 2, wherein the system classifies the wound based onthe three dimensional dataset.
 16. The biological tissue inspectionapparatus of claim 4, wherein the system classifies the burn based onthe three dimensional dataset.